|
|
|
# Testing
|
|
|
|
|
|
|
|
## Work Plan
|
|
|
|
|
|
|
|
* Maintain Selenium up-to-date and support more platforms.
|
|
|
|
* Maintain documentation of the testing framework.
|
|
|
|
* Report browser bugs upstream.
|
|
|
|
* Maintain the testsuite:
|
|
|
|
* Ensure that the tests work correctly on all platforms.
|
|
|
|
* Add tests for new MathJax issues.
|
|
|
|
* Find a way to improve simulation of user interaction via Selenium 2's "native events". Improve/Complete UI and Configuration tests that require this feature.
|
|
|
|
* Continue to improve the testing framework interface, including:
|
|
|
|
* Interface to choose/update a browser version (stable, beta, nightly etc). For the moment this is possible with Selenium 1 by installing the programs somewhere on a test machine and setting the browserPath configuration option accordingly.
|
|
|
|
* Set up machines to run tests and report errors regularly.
|
|
|
|
## General goals and regular tasks
|
|
|
|
|
|
|
|
* Improve the testing framework:
|
|
|
|
* Improve QA interface to make test execution and failure verification the easiest possible.
|
|
|
|
* Use the latest version of Selenium and support more platforms. See [[Platforms Supported]].
|
|
|
|
* Maintain our test infrastructure. See [[DSI-test-machine]].
|
|
|
|
* Maintain the test suite:
|
|
|
|
* Ensure that tests work on all platforms and that new releases of browsers or MathJax do not make them break.
|
|
|
|
* Add tests for [MathJax issues](https://github.com/mathjax/MathJax/issues?sort=updated&labels=QA+-+Unit+Test+Wanted&state=open).
|
|
|
|
* Add tests for still uncovered features.
|
|
|
|
* Keep documentation up-to-date:
|
|
|
|
* MathJax developement wiki.
|
|
|
|
* [MathJax-test documentation](http://devel.mathjax.org/testing/web/docs/html/index.html).
|
|
|
|
* [Doxygen documentation](http://devel.mathjax.org/testing/web/docs/doxygen/index.html).
|
|
|
|
* Oversee browser support evolution:
|
|
|
|
* Report browser bugs upstream.
|
|
|
|
* Verify browser improvements and report to MathJax developers.
|
|
|
|
* Keep this information up-to-date for use with browser developers or possible MathJax partners.
|
|
|
|
|
|
|
|
## Work plan
|
|
|
|
|
|
|
|
* [[Platforms Supported]]. Mainly dependent of Selenium evolutions and available infrastructure.
|
|
|
|
* improve selection mode for Internet Explorer (do not rely on Selenium 1 API)
|
|
|
|
* support for Chrome Linux with Webdriver
|
|
|
|
* support for Safari with Webdriver
|
|
|
|
* support for Android/iOS with Webdriver
|
|
|
|
* support for Konqueror with Webdriver?
|
|
|
|
* Ensure that the tests work correctly on all platforms or at least have known failures annotated. **In Progress**
|
|
|
|
* Create wiki pages to present browser bugs or missing features that are problematic for MathJax.
|
|
|
|
* Moving the MathJax testing framework into the cloud.
|
|
|
|
* Review the MathJax 2.0 doc and add missing tests.
|
|
|
|
* Add test for MathJax extensions.
|
|
|
|
* Make test comparison less strict (fuzzy reftests, avoid rounding issues etc)
|
|
|
|
* Find a way to improve simulation of user interaction via Selenium 2's "native events". Improve/Complete UI and Configuration tests that require this feature.
|
|
|
|
* Interface to choose/update a browser version (stable, beta, nightly etc). For the moment this is possible with Selenium 1 by installing the programs somewhere on a test machine and setting the browserPath configuration option accordingly.
|
|
|
|
|
|
|
|
## Work done since January 2012
|
|
|
|
|
|
|
|
* Final testing of MathJax 2.0.
|
|
|
|
* Make Webdriver work with Internet Explorer.
|
|
|
|
* Set up DSI test machine to run automated tests.
|
|
|
|
* Finishing documentation describing the testing framework, including installation instructions.
|
|
|
|
* Documentation on how to use the DSI machine.
|
|
|
|
* Allowing annotation of reftest failure.
|
|
|
|
|
|
|
|
## From May 2011 to the end of December 2011
|
|
|
|
|
|
|
|
1. Tests covering most of the LaTeX support. **Done**
|
|
|
|
2. Tests covering most of the MathML support. **Done**
|
|
|
|
3. Tests covering most of the public API. **Done**
|
|
|
|
4. Tests covering most of the Configuration options. **Done**
|
|
|
|
5. Tests covering most of the UI. **Done**
|
|
|
|
6. Web User Interface to control the testing instances. **Done** (see taskViewer.png in attachment)
|
|
|
|
7. Allow scheduling testing instances. **Done**
|
|
|
|
8. Upgrade Selenium and support more platforms. **Done** (see [[Migration from Selenium 1 to Selenium 2]] and [[Platforms Supported]])
|
|
|
|
9. Interface to choose/update a version of MathJax (stable, development, developer's branches etc). **Done**
|
|
|
|
10. Set up the webfactional machine to store the test suite, the development branches, the test runner and the QA web interface. **Done**
|
|
|
|
11. Set up DSI test machines to run automated tests. **Done**
|
|
|
|
12. <del>Connect the webfactional machine with the test machines.</del>
|
|
|
|
* Tests covering most of the LaTeX support.
|
|
|
|
* Tests covering most of the MathML support.
|
|
|
|
* Tests covering most of the public API.
|
|
|
|
* Tests covering most of the Configuration options.
|
|
|
|
* Tests covering most of the UI.
|
|
|
|
* Web User Interface to control the testing instances. (see taskViewer.png in attachment)
|
|
|
|
* Allow scheduling testing instances.
|
|
|
|
* Upgrade Selenium and support more platforms. (see [[Migration from Selenium 1 to Selenium 2]] and [[Platforms Supported]])
|
|
|
|
* Support for SVG output Jax.
|
|
|
|
* Interface to choose/update a version of MathJax (stable, development, developer's branches etc).
|
|
|
|
* Set up the webfactional machine to store the test suite, the development branches, the test runner and the QA web interface.
|
|
|
|
|
|
|
|
## From the end of January 2011 to the end of April 2011
|
|
|
|
|
|
|
|
1. Analyze areas of functionality to be tested, strategy for testing them, and identifying test cases to be created.
|
|
|
|
* Analyze areas of functionality to be tested, strategy for testing them, and identifying test cases to be created.
|
|
|
|
* See [[TestSuite]]
|
|
|
|
2. Review automated test frameworks, and select one.
|
|
|
|
* Review automated test frameworks, and select one.
|
|
|
|
* See [[Automated Test Framework]]. Our choice is Selenium. We plan to move to Selenium 2 as soon as it is released to gain support for Opera. Google is also providing Selenium [support for mobile devices](http://code.google.com/p/selenium/wiki/WebDriverForMobileBrowsers), which we will use as it becomes available.
|
|
|
|
3. Create a test plan. After preliminary discussions with Frédéric, our plan is:
|
|
|
|
* Create a test plan. After preliminary discussions with Frédéric, our plan is:
|
|
|
|
* Create Selenium ref tests for LaTeX and MathML processing, rendering, and UI testing that can be automated.
|
|
|
|
* Create "scenario tests" for installation, configuration, API and UI testing that can't easily be tested with Selenium ref tests. The scripts in the current ./test directory can be thought of as an initial step in this direction
|
|
|
|
* Define testing procedures.
|
|
|
|
4. Create a test suite of ref tests to be executed in Selenium. The plan here is:
|
|
|
|
* Create a test suite of ref tests to be executed in Selenium. The plan here is:
|
|
|
|
* Robert will setup a github repository dedicated to MathJax testing.
|
|
|
|
* <del>Robert will find people to try Selenium and the automated testing from MathJax-test github repository.</del>
|
|
|
|
* Frédéric will start writing reftests for LaTeXToMathML and MathMLToDisplay covering the main features.
|
|
|
|
* Frédéric will check the issue trackers for sensitive parts of the implementation that should be tested in priority (marked with QA tags). |
|
|
|
* Frédéric will check the issue trackers for sensitive parts of the implementation that should be tested in priority (marked with QA tags). |
|
|
\ No newline at end of file |
