| ... | ... | @@ -31,73 +31,111 @@ |
|
|
|
## AWS Management Console
|
|
|
|
|
|
|
|
* Instances
|
|
|
|
- Windows and Linux instances in use
|
|
|
|
* Windows and Linux instances in use
|
|
|
|
|
|
|
|
* Images
|
|
|
|
|
|
|
|
* AMIs: images of the machines.
|
|
|
|
- Linux
|
|
|
|
- Windows
|
|
|
|
* Linux
|
|
|
|
* Windows
|
|
|
|
|
|
|
|
* Elastic Block Store
|
|
|
|
|
|
|
|
* Volumes:
|
|
|
|
- Windows and Linux volumes of instances in use
|
|
|
|
* Windows and Linux volumes of instances in use
|
|
|
|
|
|
|
|
* Snapshots:
|
|
|
|
- Windows (30GiB)
|
|
|
|
- Linux (8GiB)
|
|
|
|
* Windows (30GiB)
|
|
|
|
* Linux (8GiB)
|
|
|
|
|
|
|
|
* Network & Security
|
|
|
|
|
|
|
|
* Security Groups
|
|
|
|
- mathjaxtest
|
|
|
|
- ICMP, TCP, UDP from mathjax-test: allow total communication between member of the security groups.
|
|
|
|
- HTTP allow connection to the Web interface
|
|
|
|
- SSH: allow connection to the Linux machine
|
|
|
|
- RDP: allow connection to the Windows machine
|
|
|
|
- The VNC port is disallowed. Use VNC over a SSH session for better security.
|
|
|
|
- The Windows machine also has its own security configuration in addition to the above setting. In particular, "10.0.0.30" is in the list of Internet Explorer trusted sites and the Windows Firewall has specific rules to accept connection from the task handler.
|
|
|
|
* mathjaxtest
|
|
|
|
* ICMP, TCP, UDP from mathjax-test: allow total communication between member of the security groups.
|
|
|
|
* HTTP allow connection to the Web interface
|
|
|
|
* SSH: allow connection to the Linux machine
|
|
|
|
* RDP: allow connection to the Windows machine
|
|
|
|
* The VNC port is disallowed. Use VNC over a SSH session for better security.
|
|
|
|
* The Windows machine also has its own security configuration in addition to the above setting. In particular, "10.0.0.30" is in the list of Internet Explorer trusted sites and the Windows Firewall has specific rules to accept connection from the task handler.
|
|
|
|
|
|
|
|
* Key Pairs
|
|
|
|
- mathjaxtest: key to use for remote access (SSH and RDP)
|
|
|
|
* mathjaxtest: key to use for remote access (SSH and RDP)
|
|
|
|
|
|
|
|
## Starting the Testing Framework
|
|
|
|
## Test Execution Overview
|
|
|
|
|
|
|
|
* It may be a good idea to check that the programs on the machines and the
|
|
|
|
testing framework are up-to-date, especially if the machines have not been
|
|
|
|
used for a long time. See "Maintenance of the Testing Machine"
|
|
|
|
|
|
|
|
* Create Amazon EC2 instances. See "Initializing the Testing Framework".
|
|
|
|
|
|
|
|
* Control the Amazon EC2 instances. See "Remote Access".
|
|
|
|
|
|
|
|
* Start the Selenium servers
|
|
|
|
* Linux: From a separate screen window, execute "~/seleniumServer.sh"
|
|
|
|
* Windows: Open "Windows PowerShell" and execute ".\seleniumServer.bat"
|
|
|
|
* If you have any problem to interrupt Selenium instances you may stop the server and run it again.
|
|
|
|
|
|
|
|
* Use the QA Web Interface to create and run tasks. See
|
|
|
|
http://devel.mathjax.org/testing/web/docs/html/qa-web-interface.html for details. TODO: give more explanation here?
|
|
|
|
|
|
|
|
* Save the test results. Terminate the Amazon EC2 instances and release elastic
|
|
|
|
IPs so that we won't be charged anymore. See "Stopping the Testing Framework".
|
|
|
|
|
|
|
|
## Maintenance of the Testing Machine
|
|
|
|
|
|
|
|
* "Micro On-Demand Instances" can be used to do maintenance tasks. They can be executed as normal Amazon EC2 instances, except that you can not use a VPC and thus can not assign an elastic IP.
|
|
|
|
|
|
|
|
* Linux
|
|
|
|
* Updating the programs: "sudo apt-get update && sudo apt-get upgrade"
|
|
|
|
* Updating the testing framework.
|
|
|
|
cd MathJax-test/
|
|
|
|
git pull
|
|
|
|
etc
|
|
|
|
|
|
|
|
* Windows
|
|
|
|
* Updating the programs: use the graphical interface.
|
|
|
|
|
|
|
|
* Preserving the state of the Linux or Windows machines
|
|
|
|
Instances Actions => Create an AMI file. After a moment, a new AMI image and Snapshot will be created. You can now terminate the instance and delete the former AMI and Snapshot. Problem: it may be easy to unintentionally delete the Snapshot of a machine.
|
|
|
|
|
|
|
|
## Initialization of the Testing Framework
|
|
|
|
|
|
|
|
* Running an Instance
|
|
|
|
- Click on Images => AMIs. Right click on an instance (Linux or Windows) and select "Launch Instance"
|
|
|
|
- Choose an Instance Type, select VPC and Continue. Suggestion: use "High-CPU On-Demand Instances" for intensive testing. "Standard On-Demand Instances" may be fine otherwise. "Micro On-Demand Instances" can also be used to do maintenance tasks (they can't be used within a VPC).
|
|
|
|
- Enter a private IP address for your instance, and Continue. The framework is configured to assume that IPs from 10.0.0.30 to 10.0.0.34 correspond to Linux machines while IPs from 10.0.0.40 to 10.0.0.44 correspond to Windows machines. Also the main machine (with the test suite, MathJax branches and Task Controller) is assumed to be 10.0.0.30.
|
|
|
|
- Skip the next screen.
|
|
|
|
- Choose the mathjaxtest Key Pairs and Continue.
|
|
|
|
- Choose the mathjaxtest Security Groups and Continue.
|
|
|
|
- Launch the instance.
|
|
|
|
* Click on Images => AMIs. Right click on an instance (Linux or Windows) and select "Launch Instance"
|
|
|
|
* Choose an Instance Type, select VPC and Continue. Suggestion: use "High-CPU On-Demand Instances" for intensive testing. "Standard On-Demand Instances" may be fine otherwise.
|
|
|
|
* Enter a private IP address for your instance, and Continue. The framework is configured to assume that IPs from 10.0.0.30 to 10.0.0.34 correspond to Linux machines while IPs from 10.0.0.40 to 10.0.0.44 correspond to Windows machines. Also the main machine (with the test suite, MathJax branches and Task Controller) is assumed to be 10.0.0.30.
|
|
|
|
* Skip the next screen.
|
|
|
|
* Choose the mathjaxtest Key Pairs and Continue.
|
|
|
|
* Choose the mathjaxtest Security Groups and Continue.
|
|
|
|
* Launch the instance.
|
|
|
|
|
|
|
|
* Associating Elastic IP to an instance
|
|
|
|
- Click on Network & Security => Elastic IPs
|
|
|
|
- Allocate New Adress, choose VPC, Yes Allocate
|
|
|
|
- Right Click on the Elastic IP => Associate
|
|
|
|
- Associate IP to an instance, Yes Associate
|
|
|
|
* Click on Network & Security => Elastic IPs
|
|
|
|
* Allocate New Adress, choose VPC, Yes Allocate
|
|
|
|
* Right Click on the Elastic IP => Associate
|
|
|
|
* Associate IP to an instance, Yes Associate
|
|
|
|
|
|
|
|
## Remote Access
|
|
|
|
|
|
|
|
* Some SSH/VNC/RDP clients
|
|
|
|
- Linux: vinagre (SSH, VNC), rdeskop (RDP)
|
|
|
|
- Windows, Mac: ?
|
|
|
|
* Linux: vinagre (SSH, VNC), rdeskop (RDP)
|
|
|
|
* Windows, Mac: ?
|
|
|
|
|
|
|
|
* Linux Instance
|
|
|
|
- Use a SSH client to connect to the instance.
|
|
|
|
- Use the "screen" command to handle several windows. Quick guide to the screen environment: use "Ctrl + a, Ctrl + c" to create a new window, "Ctrl + a, Ctrl + n" to move to the next window, "Ctrl + a, Ctrl + p" to move to the previous window, use the "exit" command to close the current window and finally "Ctrl + a, Ctrl + d" to detach the screen session. When you detach a screen session, all the servers will keep running even if you logout from the SSH session. You can restore your screen session when you connect again with the command "screen -r" (provided the machine has not been stopped since your last connection).
|
|
|
|
- In one screen window, use "~/start.sh" to start the VNC server. On the machine 10.0.0.30, this will also mount the mathjaxtest bucket on ~/MathJax-results/ and start the task handler.
|
|
|
|
- If necessary, you may use a VNC client to connect to the instance.
|
|
|
|
* Use a SSH client to connect to the instance.
|
|
|
|
* Use the "screen" command to handle several windows. Quick guide to the screen environment: use "Ctrl + a, Ctrl + c" to create a new window, "Ctrl + a, Ctrl + n" to move to the next window, "Ctrl + a, Ctrl + p" to move to the previous window, use the "exit" command to close the current window and finally "Ctrl + a, Ctrl + d" to detach the screen session. When you detach a screen session, all the servers will keep running even if you logout from the SSH session. You can restore your screen session when you connect again with the command "screen -r" (provided the machine has not been stopped since your last connection).
|
|
|
|
* In one screen window, use "~/start.sh" to start the VNC server. On the machine 10.0.0.30, this will also mount the mathjaxtest bucket on ~/MathJax-results/ and start the task handler.
|
|
|
|
* If necessary, you may use a VNC client to connect to the instance.
|
|
|
|
|
|
|
|
* Windows Instance
|
|
|
|
- Use a RDP client to connect to the instance.
|
|
|
|
- Open a Windows Session.
|
|
|
|
* Use a RDP client to connect to the instance.
|
|
|
|
* Open a Windows Session.
|
|
|
|
|
|
|
|
* Web Interface
|
|
|
|
- Open the page http://{public address of the main machine}/testing/web/
|
|
|
|
- Read http://devel.mathjax.org/testing/web/docs/html/qa-web-interface.html for more details.
|
|
|
|
* Open the page http://{public address of the main machine}/testing/web/
|
|
|
|
* Read http://devel.mathjax.org/testing/web/docs/html/qa-web-interface.html for more details.
|
|
|
|
|
|
|
|
* Directory Structure
|
|
|
|
* Linux
|
| ... | ... | @@ -110,36 +148,19 @@ |
|
|
|
* ~\selenium-server.jar: Selenium server.
|
|
|
|
* ~\Desktop\Webdrivers\: Drivers for Chrome and Internet Explorer. Added to $PATH.
|
|
|
|
|
|
|
|
* Installation (see [[Platform Supported]])
|
|
|
|
* Installation (see [[Platforms Supported]])
|
|
|
|
* Browsers: Firefox, Chrome, Internet Explorer, Safari, Opera, Konqueror.
|
|
|
|
* Fonts: STIX and Mathjax fonts.
|
|
|
|
* Plugin: MathPlayer
|
|
|
|
* Selenium Servers (see directory structure)
|
|
|
|
* Webdrivers (see directory structure)
|
|
|
|
|
|
|
|
# Running the Tests
|
|
|
|
|
|
|
|
* Starting Selenium servers
|
|
|
|
- Linux: From a separate screen window, execute "~/seleniumServer.sh"
|
|
|
|
- Windows: Open "Windows PowerShell" and execute ".\seleniumServer.bat"
|
|
|
|
- If you have any problem to interrupt Selenium instances you may stop the server and run it again.
|
|
|
|
|
|
|
|
* Creating task
|
|
|
|
|
|
|
|
* Running task
|
|
|
|
|
|
|
|
# Stopping the Testing Framework
|
|
|
|
## Stopping the Testing Framework
|
|
|
|
|
|
|
|
* Saving the test results to Amazon S3
|
|
|
|
The test results are saved on the main machine in MathJax-test/web/results/. They will be cleared when the instance terminates. To save a copy to Amazon S3, just copy the relevant files in ~/MathJax-results/.
|
|
|
|
|
|
|
|
* Terminating an Instance
|
|
|
|
Warning: this will totally clear the instance and its associated volume. Be sure to save unfinished task before doing so. In particular save a copy of the test results to Amazon S3. See also "Maintenance of the Testing Machine" if you need to update the AMIs.
|
|
|
|
- Right Click on an instance and select terminate.
|
|
|
|
- Check "Release Elastic IPs", Yes Terminate
|
|
|
|
|
|
|
|
# Maintenance of the Testing Machine
|
|
|
|
|
|
|
|
* Preserving the state of the Linux or Windows machines
|
|
|
|
Instances Actions => Create an AMI file. After a moment, a new AMI image and Snapshot will be created. You can now terminate the instance and delete the former AMI and Snapshot. Problem: it may be easy to unintentionally delete the Snapshot of a machine.
|
|
|
|
|
|
|
|
* Right Click on an instance and select terminate.
|
|
|
|
* Check "Release Elastic IPs", Yes Terminate |
