|
|
Test Machines
|
|
|
# Test Machines
|
|
|
|
|
|
Todo...
|
|
|
## Overview
|
|
|
|
|
|
It is highly recommended to read the
|
|
|
[MathJax-test documentation](http://devel.mathjax.org/testing/web/index.php)
|
|
|
to understand how the [Test Framework](https://github.com/mathjax/MathJax-test/)
|
|
|
is working. Our test infrastructure contains several installations of the
|
|
|
test framework and they are managed the same way. For each installation we have
|
|
|
a "main" machine, that can be used to control the test on machines of the same
|
|
|
local network (including the main machine itself). The test results are stored
|
|
|
stored on the main machine and can be made public. Here is the how our
|
|
|
infrastructure is organized:
|
|
|
|
|
|
* [Public test results](http://mathjaxtest.s3.amazonaws.com/index.html) are
|
|
|
stored on Amazon S3.
|
|
|
* A test framework is installed on Amazon EC2 and is only used during the test
|
|
|
period. A "micro" Linux instance is continuously running during that
|
|
|
period and contains an Apache installation to serve
|
|
|
[public Web pages](http://devel.mathjax.org/testing/web/index.php)
|
|
|
with the MathJax test suite, QA tools and documentation. More powerful
|
|
|
Linux or Windows instances can be executed when we want to run the tests.
|
|
|
The "micro" instance controls the execution on the other machines, collects
|
|
|
the results and can be used to publish these results on Amazon S3.
|
|
|
* A DSI Mac machine located in Long Beach, where the test framework is
|
|
|
installed.
|
|
|
* A DSI Mac machine located in St. Paul. It contains two partitions (Mac and
|
|
|
Windows) with a test framework installed on each of them. The Windows
|
|
|
partition also contains a virtual Linux machine which can be controlled by
|
|
|
the Windows machine.
|
|
|
* The member of the MathJax team may also install the test framework on their
|
|
|
own machine.
|
|
|
|
|
|
As a rule of thumb, use the test framework on your local machine for simple
|
|
|
testing: you can either verify the tests manually or automatically.
|
|
|
For intensive testing, use the Amazon EC2 instance whose remote connection is
|
|
|
more reliable, where you can run several machines of different CPU powers and
|
|
|
from which you can directly publish the results to Amazon S3. Some drawbacks are
|
|
|
that the instances do not have fixed IPs and we have to pay the service (running
|
|
|
instances, server communications etc). Finally, use the DSI machine for Mac
|
|
|
testing or when we are outside the test period. Note that the remote interface
|
|
|
is less convenient and slower and the machines may not always be available.
|
|
|
Also, the tests results won't be gathered at the same place.
|
|
|
|
|
|
## Maintenance
|
|
|
|
|
|
MathJax, the browsers, Selenium, the test framework etc are always
|
|
|
Ideally, we would like to use only EC2 instances created from a few
|
|
|
disk images so that we only need to maintain these disk images. That's not
|
|
|
currently the case so we have to keep each test machine up-to-date. Fortunately,
|
|
|
the browsers, operating systems and test framework generally have tools to do
|
|
|
this task more or less easily. Here is an overview:
|
|
|
|
|
|
1. Updating the programs on the operating system
|
|
|
* On Linux, use the command "sudo apt-get update && sudo apt-get upgrade"
|
|
|
* On Mac, use the command "sudo port selfupdate" to update the MacPorts
|
|
|
packages. You may also need to use the graphical package managers of the
|
|
|
system or of third-party tools.
|
|
|
* On Windows use Windows update or other graphical package managers of
|
|
|
third-party tools.
|
|
|
|
|
|
2. Updating programs used by test machines
|
|
|
* Browsers: This is done by the system package manager on Linux and on Windows
|
|
|
Most browsers now have their own integrated automatic updates.
|
|
|
* Selenium Server: if the test machine has the test framework installed, you
|
|
|
can use the "make downloadSeleniumServer" command (you may need to override
|
|
|
the version specified in custom.cfg). Otherwise, you must get it manually
|
|
|
from the [Selenium download page](http://seleniumhq.org/download/).
|
|
|
* Browser drivers: Only for Internet Explorer and Chrome. The Internet
|
|
|
Explorer driver is available on the
|
|
|
[Selenium download page](http://seleniumhq.org/download/) while the Chrome
|
|
|
driver can be found on the
|
|
|
[ChromeDriver download page](http://code.google.com/p/chromedriver/downloads/list)
|
|
|
* Java: This is done by the system package manager on Linux and there should
|
|
|
be a graphical updater on other platforms.
|
|
|
* STIX and MathJax fonts: the STIX fonts should be kept up-to-date by the
|
|
|
system package manager. Otherwise you can find package and installers
|
|
|
on [Mozilla MathML Fonts page](https://developer.mozilla.org/en-US/docs/Mozilla_MathML_Project/Fonts)
|
|
|
* MathPlayer: Updates should be checked automatically when you open a page
|
|
|
with MathML in Internet Explorer.
|
|
|
|
|
|
3. Updating the test framework
|
|
|
* Open a UNIX terminal and go in MathJax-test/
|
|
|
* Backup the default configuration: "cp default.cfg default-.cfg"
|
|
|
* Get the update: "git pull"
|
|
|
* Verify whether the format of the configuration file has changed:
|
|
|
"diff default.cfg default-.cfg". If so, then you will have to add/remove
|
|
|
configuration options in custom.cfg accordingly. Hopefully, this format won't
|
|
|
change too often.
|
|
|
* "make config"
|
|
|
* You may also want to update the documentation ("make doc"), the
|
|
|
selenium driver ("sudo make updateSeleniumDriver") or the MathJax branches
|
|
|
("make updateMathJaxBranches"). If necessary, execute "make config" again.
|
|
|
|
|
|
4. Saving the AMI.
|
|
|
For Amazon EC2 machines, your changes will be lost if you terminate an
|
|
|
instance. To preserve the state of the Linux or Windows machines, do
|
|
|
Instances Actions => Create an AMI file. After a moment, a new AMI image and
|
|
|
Snapshot will be created. You can now terminate the instance and delete the
|
|
|
former AMI and Snapshot. Be careful not to delete the wrong Snapshot!
|
|
|
|
|
|
## See also
|
|
|
|
|
|
(Need to be updated and expanded)
|
|
|
|
|
|
[[Amazon-EC2]] and [[DSI-test-machine]] |
|
|
\ No newline at end of file |