... | @@ -42,6 +42,26 @@ testing or when we are outside the test period. Note that the remote interface |
... | @@ -42,6 +42,26 @@ testing or when we are outside the test period. Note that the remote interface |
|
is less convenient and slower and the machines may not always be available.
|
|
is less convenient and slower and the machines may not always be available.
|
|
Also, the tests results won't be gathered at the same place.
|
|
Also, the tests results won't be gathered at the same place.
|
|
|
|
|
|
|
|
## Remote access
|
|
|
|
|
|
|
|
Use remote access to maintain our test framework infrastructure or to execute
|
|
|
|
tests on the machines. Please read the "Test Machines" document from our
|
|
|
|
internal documentation for more details.
|
|
|
|
|
|
|
|
When using SSH, the "screen" command is very useful to handle several windows.
|
|
|
|
Here is a quick guide to the screen environment:
|
|
|
|
|
|
|
|
* "Ctrl + a, Ctrl + c" to create a new window
|
|
|
|
* "Ctrl + a, Ctrl + n" to move to the next window and
|
|
|
|
* "Ctrl + a, Ctrl + p" to move to the previous window
|
|
|
|
* Type "exit" to close the current window
|
|
|
|
* "Ctrl + a, Ctrl + d" to detach the screen session.
|
|
|
|
|
|
|
|
When you detach a screen session, all the servers will keep running even if you
|
|
|
|
logout from the SSH session. You can restore your screen session when you
|
|
|
|
connect again with the command "screen -r" (provided the machine has not been
|
|
|
|
stopped since your last connection).
|
|
|
|
|
|
## Maintenance
|
|
## Maintenance
|
|
|
|
|
|
MathJax, the browsers, Selenium, the test framework etc are always
|
|
MathJax, the browsers, Selenium, the test framework etc are always
|
... | @@ -99,6 +119,103 @@ this task more or less easily. Here is an overview: |
... | @@ -99,6 +119,103 @@ this task more or less easily. Here is an overview: |
|
Snapshot will be created. You can now terminate the instance and delete the
|
|
Snapshot will be created. You can now terminate the instance and delete the
|
|
former AMI and Snapshot. Be careful not to delete the wrong Snapshot!
|
|
former AMI and Snapshot. Be careful not to delete the wrong Snapshot!
|
|
|
|
|
|
|
|
## Starting/Terminating Amazon EC2 instance
|
|
|
|
|
|
|
|
To run an instance go to AWS Management Console and follow these steps
|
|
|
|
|
|
|
|
* Click on Images => AMIs. Right click on an instance (Linux or Windows) and
|
|
|
|
select "Launch Instance"
|
|
|
|
* Choose an Instance Type and Continue. Suggestion: use
|
|
|
|
"Micro On-Demand Instances" for the main Linux machine or for maintenance task
|
|
|
|
and
|
|
|
|
"Small Standard On-Demand Instances" or "Medium High-CPU On-Demand Instances"
|
|
|
|
for the test machines. You might want to run the Internet Explorer tests
|
|
|
|
separately with a more powerful machine.
|
|
|
|
* Skip dialog until the "Key Pairs", choose the mathjaxtest Key Pairs
|
|
|
|
and Continue.
|
|
|
|
* Choose the mathjaxtest Security Groups and Continue.
|
|
|
|
* Launch the instance.
|
|
|
|
|
|
|
|
Once started you can check Instances => Instances to check the public and
|
|
|
|
private IP/hostname of each instance (TODO: give more details).
|
|
|
|
One issue is that these addresses are
|
|
|
|
created when the instance is created and thus are not fixed. There are
|
|
|
|
workarounds:
|
|
|
|
|
|
|
|
* For the public address, you can create elastic IP that won't change until
|
|
|
|
they are explicitly released. Note that elastic IPs are free only if they are
|
|
|
|
associated to a running instance, so it is worth to use one for the main Linux
|
|
|
|
machine only (the other instances are terminated after the test executions).
|
|
|
|
To allocate, associate and release an elastic IP, check the Network &
|
|
|
|
Security => Elastic IPs.
|
|
|
|
|
|
|
|
* For the private address, you can use a VPC. But we don't do this since this
|
|
|
|
is not available with "micro" instances and VPC and non-VPC instances can
|
|
|
|
not communicate.
|
|
|
|
|
|
|
|
This basically means that for remote connection, you will need to specify a
|
|
|
|
public address that change change (except for the main Linux machine).
|
|
|
|
Similarly, for test executions, you will need to specify a private address that
|
|
|
|
may change. Together with the cost, this is one of serious drawback of the
|
|
|
|
Amazon EC2 infrastructure.
|
|
|
|
|
|
|
|
To terminate an instance, right click on an instance and select terminate.
|
|
|
|
Check "Release Elastic IPs", Yes Terminate. Warning: this will totally clear
|
|
|
|
the instance and its associated volume. Be sure to save unfinished task before
|
|
|
|
doing so. In particular save a copy of the test results to Amazon S3. See also
|
|
|
|
"Maintenance of the Testing Machine" if you need to update the AMIs.
|
|
|
|
|
|
|
|
## Test period
|
|
|
|
|
|
|
|
A test period takes place before each MathJax release. During that period,
|
|
|
|
a Linux instance is used to control the execution of tests, to copy the results
|
|
|
|
to Amazon S3 and to serve http://devel.mathjax.org/.
|
|
|
|
|
|
|
|
At the beginning of the test period, follow the "Starting Amazon EC2 instance"
|
|
|
|
instruction to start a "micro" Linux instance. Allocate an elastic IP and attach
|
|
|
|
it to the instance. In the Go Daddy interface, make
|
|
|
|
http://devel.mathjax.org/ point to this elastic IP address.
|
|
|
|
|
|
|
|
At the end of the test period, stop the instance and do not forget to release
|
|
|
|
the elastic IP and do other cleanup so we won't be charged anymore. In the
|
|
|
|
EC2 console Dashboard you should only see "2 EBS Snapshots" (Windows and Linux)
|
|
|
|
but "0 Running Instances", "0 Elastic IPs" and "0 EBS Volumes". You may also
|
|
|
|
use Go Daddy to remove the devel.mathjax.org mapping.
|
|
|
|
|
|
|
|
## Before running the tests
|
|
|
|
|
|
|
|
It may be a good idea to check that the programs on the machines and the
|
|
|
|
testing framework are up-to-date, especially if the machines have not been used
|
|
|
|
for a long time. See "Maintenance of the Testing Machine".
|
|
|
|
|
|
|
|
Next, you must start the machines and determine their private IPs or host names:
|
|
|
|
|
|
|
|
* On Amazon EC2 see "Starting Amazon EC2 instance" above. The main Linux
|
|
|
|
machine can be configured with the help of the script set-task-handler-ip.sh.
|
|
|
|
For the test machines, use the private DNS given in the AWS Management Console.
|
|
|
|
|
|
|
|
* Otherwise, you have to determine the IPs manually. When you have only one
|
|
|
|
machine which contains the test framework and on which the tests are executed,
|
|
|
|
you can just use "localhost" or 127.0.0.1. On Linux/Mac
|
|
|
|
you can use the command "hostname", "hostname -I" or "sudo ifconfig" to get
|
|
|
|
that information. On Windows, "ipconfig" should do the job.
|
|
|
|
|
|
|
|
On one terminal (or "screen" windows for Amazon EC2) of the main machine, run
|
|
|
|
the task handler with "make runTaskHandler". You might want to run
|
|
|
|
"make clearTaskList" before, to clean up a previous set up.
|
|
|
|
|
|
|
|
On each test machine, open a terminal and run the selenium server. On Linux EC2,
|
|
|
|
it is recommended to open a terminal from the GUI interface.
|
|
|
|
(TODO: give more details)
|
|
|
|
|
|
|
|
## Running the tests
|
|
|
|
|
|
|
|
...
|
|
|
|
|
|
|
|
## After running the tests
|
|
|
|
|
|
|
|
...
|
|
|
|
|
|
## See also
|
|
## See also
|
|
|
|
|
|
(Need to be updated and expanded)
|
|
(Need to be updated and expanded)
|
... | | ... | |