Hi
Dne Tue, 22 Apr 2014 08:17:07 -0400 Marc Delisle marc@infomarc.info napsal(a):
For me, it's difficult to see when these failures are real or not. For example, after your patch revert
https://ci.phpmyadmin.net/job/phpMyAdmin-continuous/6120/
there were a few Selenium failures. Are these real?
How you define real bugs? If you mean bugs in our code, then almost all bugs are real (basically all with exception of timeouts when talking with Selenium server), however some of these are bugs in our Selenium tests. Most of these are some timing issues like the Selenium driver is clicking on elements too early (eg. before Javascript initialization is completed).
These are less visible (happen less often) on Jenkins and more visible on Travis (due to higher latencies caused by local proxy used for testing).
I've already fixed many of such errors, but I somehow ran out of time I can spend on this, so any help is welcome here.
The ideal situation would be to fix these tests completely and do strict Selenium testing in Travis as well to have checked all pull requests with it...