... | @@ -3,11 +3,14 @@ |
... | @@ -3,11 +3,14 @@ |
|
## General goals and regular tasks
|
|
## General goals and regular tasks
|
|
|
|
|
|
* Improve the testing framework:
|
|
* Improve the testing framework:
|
|
* Improve QA interface to make test execution and failure verification the easiest possible.
|
|
* Improve QA interface to make test execution and failure verification the
|
|
* Use the latest version of Selenium and support more platforms. See [[Platforms Supported]].
|
|
easiest possible.
|
|
|
|
* Use the latest version of Selenium and support more platforms. See
|
|
|
|
[[Platforms Supported]].
|
|
* Maintain our test infrastructure. See [[DSI-test-machine]].
|
|
* Maintain our test infrastructure. See [[DSI-test-machine]].
|
|
* Maintain the test suite:
|
|
* Maintain the test suite:
|
|
* Ensure that tests work on all platforms and that new releases of browsers or MathJax do not make them break.
|
|
* Ensure that tests work on all platforms and that new releases of browsers
|
|
|
|
or MathJax do not make them break.
|
|
* Add tests for [MathJax issues](https://github.com/mathjax/MathJax/issues?sort=updated&labels=QA+-+Unit+Test+Wanted&state=open).
|
|
* Add tests for [MathJax issues](https://github.com/mathjax/MathJax/issues?sort=updated&labels=QA+-+Unit+Test+Wanted&state=open).
|
|
* Add tests for still uncovered features.
|
|
* Add tests for still uncovered features.
|
|
* Keep documentation up-to-date:
|
|
* Keep documentation up-to-date:
|
... | @@ -17,31 +20,42 @@ |
... | @@ -17,31 +20,42 @@ |
|
* Oversee browser support evolution:
|
|
* Oversee browser support evolution:
|
|
* Report browser bugs upstream.
|
|
* Report browser bugs upstream.
|
|
* Verify browser improvements and report to MathJax developers.
|
|
* Verify browser improvements and report to MathJax developers.
|
|
* Keep this information up-to-date for use with browser developers or possible MathJax partners.
|
|
* Keep this information up-to-date for use with browser developers or possible
|
|
|
|
MathJax partners. See [[Browser Bugs]] and [[MathML Support in Browsers]].
|
|
|
|
|
|
## Work plan
|
|
## Work plan
|
|
|
|
|
|
* [[Platforms Supported]]. Mainly dependent of Selenium evolutions and available infrastructure.
|
|
* [[Platforms Supported]]. Mainly dependent of Selenium evolutions and
|
|
|
|
available infrastructure.
|
|
* improve selection mode for Internet Explorer (do not rely on Selenium 1 API)
|
|
* improve selection mode for Internet Explorer (do not rely on Selenium 1 API)
|
|
* support for Chrome Linux with Webdriver
|
|
* support for Chrome Linux with Webdriver
|
|
* support for Safari with Webdriver
|
|
* support for Safari with Webdriver
|
|
* support for Android/iOS with Webdriver
|
|
* support for Android/iOS with Webdriver
|
|
* support for Konqueror with Webdriver?
|
|
* support for Konqueror with Webdriver?
|
|
* Ensure that the tests work correctly on all platforms or at least have known failures annotated. **In Progress**
|
|
* Ensure that the tests work correctly on all platforms or at least have known
|
|
* Create wiki pages to present browser bugs or missing features that are problematic for MathJax.
|
|
failures annotated. **In Progress**
|
|
|
|
* Create wiki pages to present browser bugs or missing features that are
|
|
|
|
problematic for MathJax.
|
|
* Moving the MathJax testing framework into the cloud.
|
|
* Moving the MathJax testing framework into the cloud.
|
|
* Review the MathJax 2.0 doc and add missing tests.
|
|
* Review the MathJax 2.0 doc and add missing tests.
|
|
* Add test for MathJax extensions.
|
|
* Add test for MathJax extensions.
|
|
* Make test comparison less strict (fuzzy reftests, avoid rounding issues etc)
|
|
* Make test comparison less strict (fuzzy reftests, avoid rounding issues etc)
|
|
* Find a way to improve simulation of user interaction via Selenium 2's "native events". Improve/Complete UI and Configuration tests that require this feature.
|
|
* Find a way to improve simulation of user interaction via Selenium 2's "native
|
|
* Interface to choose/update a browser version (stable, beta, nightly etc). For the moment this is possible with Selenium 1 by installing the programs somewhere on a test machine and setting the browserPath configuration option accordingly.
|
|
events". Improve/Complete UI and Configuration tests that require this
|
|
|
|
feature.
|
|
|
|
* Interface to choose/update a browser version (stable, beta, nightly etc). For
|
|
|
|
the moment this is possible with Selenium 1 by installing the programs
|
|
|
|
somewhere on a test machine and setting the browserPath configuration option
|
|
|
|
accordingly.
|
|
|
|
* Tests covering most of the AsciiMath support.
|
|
|
|
|
|
## Work done since January 2012
|
|
## Work done since January 2012
|
|
|
|
|
|
* Final testing of MathJax 2.0.
|
|
* Final testing of MathJax 2.0.
|
|
* Make Webdriver work with Internet Explorer.
|
|
* Make Webdriver work with Internet Explorer.
|
|
* Set up DSI test machine to run automated tests.
|
|
* Set up DSI test machine to run automated tests.
|
|
* Finishing documentation describing the testing framework, including installation instructions.
|
|
* Finishing documentation describing the testing framework, including
|
|
|
|
installation instructions.
|
|
* Documentation on how to use the DSI machine.
|
|
* Documentation on how to use the DSI machine.
|
|
* Allowing annotation of reftest failure.
|
|
* Allowing annotation of reftest failure.
|
|
|
|
|
... | @@ -52,25 +66,41 @@ |
... | @@ -52,25 +66,41 @@ |
|
* Tests covering most of the public API.
|
|
* Tests covering most of the public API.
|
|
* Tests covering most of the Configuration options.
|
|
* Tests covering most of the Configuration options.
|
|
* Tests covering most of the UI.
|
|
* Tests covering most of the UI.
|
|
* Web User Interface to control the testing instances. (see taskViewer.png in attachment)
|
|
* Web User Interface to control the testing instances. (see taskViewer.png in
|
|
|
|
attachment)
|
|
* Allow scheduling testing instances.
|
|
* Allow scheduling testing instances.
|
|
* Upgrade Selenium and support more platforms. (see [[Migration from Selenium 1 to Selenium 2]] and [[Platforms Supported]])
|
|
* Upgrade Selenium and support more platforms. (see
|
|
|
|
[[Migration from Selenium 1 to Selenium 2]] and [[Platforms Supported]])
|
|
* Support for SVG output Jax.
|
|
* Support for SVG output Jax.
|
|
* Interface to choose/update a version of MathJax (stable, development, developer's branches etc).
|
|
* Interface to choose/update a version of MathJax (stable, development,
|
|
* Set up the webfactional machine to store the test suite, the development branches, the test runner and the QA web interface.
|
|
developer's branches etc).
|
|
|
|
* Set up the webfactional machine to store the test suite, the development
|
|
|
|
branches, the test runner and the QA web interface.
|
|
|
|
|
|
## From the end of January 2011 to the end of April 2011
|
|
## From the end of January 2011 to the end of April 2011
|
|
|
|
|
|
* Analyze areas of functionality to be tested, strategy for testing them, and identifying test cases to be created.
|
|
* Analyze areas of functionality to be tested, strategy for testing them, and
|
|
|
|
identifying test cases to be created.
|
|
* See [[TestSuite]]
|
|
* See [[TestSuite]]
|
|
* Review automated test frameworks, and select one.
|
|
* Review automated test frameworks, and select one.
|
|
* See [[Automated Test Framework]]. Our choice is Selenium. We plan to move to Selenium 2 as soon as it is released to gain support for Opera. Google is also providing Selenium [support for mobile devices](http://code.google.com/p/selenium/wiki/WebDriverForMobileBrowsers), which we will use as it becomes available.
|
|
* See [[Automated Test Framework]]. Our choice is Selenium. We plan to move
|
|
|
|
to Selenium 2 as soon as it is released to gain support for Opera. Google is
|
|
|
|
also providing Selenium
|
|
|
|
[support for mobile devices](http://code.google.com/p/selenium/wiki/WebDriverForMobileBrowsers),
|
|
|
|
which we will use as it becomes available.
|
|
* Create a test plan. After preliminary discussions with Frédéric, our plan is:
|
|
* Create a test plan. After preliminary discussions with Frédéric, our plan is:
|
|
* Create Selenium ref tests for LaTeX and MathML processing, rendering, and UI testing that can be automated.
|
|
* Create Selenium ref tests for LaTeX and MathML processing, rendering, and
|
|
* Create "scenario tests" for installation, configuration, API and UI testing that can't easily be tested with Selenium ref tests. The scripts in the current ./test directory can be thought of as an initial step in this direction
|
|
UI testing that can be automated.
|
|
|
|
* Create "scenario tests" for installation, configuration, API and UI
|
|
|
|
testing that can't easily be tested with Selenium ref tests. The scripts in
|
|
|
|
the current ./test directory can be thought of as an initial step in this
|
|
|
|
direction
|
|
* Define testing procedures.
|
|
* Define testing procedures.
|
|
* Create a test suite of ref tests to be executed in Selenium. The plan here is:
|
|
* Create a test suite of ref tests to be executed in Selenium. The plan here is:
|
|
* Robert will setup a github repository dedicated to MathJax testing.
|
|
* Robert will setup a github repository dedicated to MathJax testing.
|
|
* <del>Robert will find people to try Selenium and the automated testing from MathJax-test github repository.</del>
|
|
* <del>Robert will find people to try Selenium and the automated testing
|
|
* Frédéric will start writing reftests for LaTeXToMathML and MathMLToDisplay covering the main features.
|
|
from MathJax-test github repository.</del>
|
|
* Frédéric will check the issue trackers for sensitive parts of the implementation that should be tested in priority (marked with QA tags). |
|
* Frédéric will start writing reftests for LaTeXToMathML and MathMLToDisplay
|
|
\ No newline at end of file |
|
covering the main features.
|
|
|
|
* Frédéric will check the issue trackers for sensitive parts of the
|
|
|
|
implementation that should be tested in priority (marked with QA tags). |
|
|
|
\ No newline at end of file |