Hi,
I've played around with phpunit a bit as of late, and think we could make better use of this for pma.
The PMA_formatNumberByteDown_test test still fails for me on a conversion.... does this pass on other systems?
I've setup several new testcases that uses Selenium RC [1] to replay actual browser behaviour. My first test in this is a case where I log in and do some basic browsing and change the test users password. The test scenario's are easily recorded using the Firefox Selenium IDE add-on. [2] Testing can happen in Firefox, Internet Explorer and other browsers.
I've set up a cronjob which would run the test suite e.g. every day on trunk, and plan to expand this to the latest testing branch. Any errors would be mailed to me. If others are interested I could sent this to a list address.
Issues: - Using a self-signed ssl certificate with Selenium RC is a pain, using non-ssl for now. [4]
- Somehow selenium does not like the language selection on the cookie login page, forced English in my config for now.
- The current naming scheme for test cases seems to be a bit off, according to the phpunit defaults. When running through wui.php it works, but when running individual tests the underscores in the class/file names are interpreted as directory separator. I therefore propose a CamelCase format for the new selenium classes.
- The wiki has no info about testing, I propose a Devel:UnitTesting page
I hope to soon commit these changes to trunk, any feedback would be very welcome.
References: 1. http://seleniumhq.org/ 2. https://addons.mozilla.org/en-US/firefox/addon/2079 2 .http://www.phpunit.de/manual/3.1/en/selenium.html 4. http://townx.org/blog/elliot/dealing-self-signed-ssl-certificates-when-runni...
Hi
Dne Wed, 15 Jul 2009 05:05:46 +0200 Herman van Rink rink@initfour.nl napsal(a):
The PMA_formatNumberByteDown_test test still fails for me on a conversion.... does this pass on other systems?
Six tests do fail for me:
1) testEncrypt(PMA_blowfish_test) 2) testDecrypt(PMA_blowfish_test) 3) testFormatNumber(PMA_formatNumberByteDown_test) with data set #4 4) testFormatNumber(PMA_formatNumberByteDown_test) with data set #5 5) testFormatNumber(PMA_formatNumberByteDown_test) with data set #6 6) testWriteReloadNavigation(PMA_headerLocation_test)
I've setup several new testcases that uses Selenium RC [1] to replay actual browser behaviour. My first test in this is a case where I log in and do some basic browsing and change the test users password. The test scenario's are easily recorded using the Firefox Selenium IDE add-on. [2] Testing can happen in Firefox, Internet Explorer and other browsers.
I've set up a cronjob which would run the test suite e.g. every day on trunk, and plan to expand this to the latest testing branch. Any errors would be mailed to me. If others are interested I could sent this to a list address.
I think sending failures to this list would be appropriate, what do others think?
Issues:
- Using a self-signed ssl certificate with Selenium RC is a pain, using
non-ssl for now. [4]
- Somehow selenium does not like the language selection on the cookie
login page, forced English in my config for now.
- The current naming scheme for test cases seems to be a bit off,
according to the phpunit defaults. When running through wui.php it works, but when running individual tests the underscores in the class/file names are interpreted as directory separator. I therefore propose a CamelCase format for the new selenium classes.
Feel free to adjust phpunit test also.
- The wiki has no info about testing, I propose a Devel:UnitTesting page
Yes, testing is completely uncovered by our docs.
Michal Čihař wrote:
Hi
Dne Wed, 15 Jul 2009 05:05:46 +0200 Herman van Rink rink@initfour.nl napsal(a):
The PMA_formatNumberByteDown_test test still fails for me on a conversion.... does this pass on other systems?
Six tests do fail for me:
- testEncrypt(PMA_blowfish_test)
- testDecrypt(PMA_blowfish_test)
These have never failed me. Could you look into this?
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #4
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #5
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #6
These 3 fail me as well, using the data: array(0.00003, 3, 2, '0,03 m'), array(0.003, 3, 3, '0,003 '), array(-0.003, 6, 0, '-3 m'),
Could someone with intimate formatting knowledge take a look at this to determine if PMA_formatNumber() needs fixing, or the testcase?
- testWriteReloadNavigation(PMA_headerLocation_test)
Should be fixed now.
- The wiki has no info about testing, I propose a Devel:UnitTesting page
Yes, testing is completely uncovered by our docs.
A first draft is available on: http://wiki.phpmyadmin.net/pma/Devel:UnitTesting
Herman van Rink a écrit :
Michal Čihař wrote:
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #4
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #5
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #6
These 3 fail me as well, using the data: array(0.00003, 3, 2, '0,03 m'), array(0.003, 3, 3, '0,003 '), array(-0.003, 6, 0, '-3 m'),
Could someone with intimate formatting knowledge take a look at this to determine if PMA_formatNumber() needs fixing, or the testcase?
I'll have a look.
Marc Delisle a écrit :
Herman van Rink a écrit :
Michal Čihař wrote:
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #4
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #5
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #6
These 3 fail me as well, using the data: array(0.00003, 3, 2, '0,03 m'), array(0.003, 3, 3, '0,003 '), array(-0.003, 6, 0, '-3 m'),
Could someone with intimate formatting knowledge take a look at this to determine if PMA_formatNumber() needs fixing, or the testcase?
I'll have a look.
Replacing PMA_pow() by pow() in PMA_formatNumber() removes the failures. Now I'm looking why...
Marc Delisle a écrit :
Marc Delisle a écrit :
Herman van Rink a écrit :
Michal Čihař wrote:
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #4
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #5
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #6
These 3 fail me as well, using the data: array(0.00003, 3, 2, '0,03 m'), array(0.003, 3, 3, '0,003 '), array(-0.003, 6, 0, '-3 m'),
Could someone with intimate formatting knowledge take a look at this to determine if PMA_formatNumber() needs fixing, or the testcase?
I'll have a look.
Replacing PMA_pow() by pow() in PMA_formatNumber() removes the failures. Now I'm looking why...
PMA_pow() exits if the exponent is negative; removing this test "fixes" the problem but I'm looking for all impacts.
Marc Delisle a écrit :
Marc Delisle a écrit :
Marc Delisle a écrit :
Herman van Rink a écrit :
Michal Čihař wrote:
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #4
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #5
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #6
These 3 fail me as well, using the data: array(0.00003, 3, 2, '0,03 m'), array(0.003, 3, 3, '0,003 '), array(-0.003, 6, 0, '-3 m'),
Could someone with intimate formatting knowledge take a look at this to determine if PMA_formatNumber() needs fixing, or the testcase?
I'll have a look.
Replacing PMA_pow() by pow() in PMA_formatNumber() removes the failures. Now I'm looking why...
PMA_pow() exits if the exponent is negative; removing this test "fixes" the problem but I'm looking for all impacts.
Should be fixed now.
Hi
Dne Thu, 16 Jul 2009 17:02:21 +0200 Herman van Rink rink@initfour.nl napsal(a):
Michal Čihař wrote:
Hi
Dne Wed, 15 Jul 2009 05:05:46 +0200 Herman van Rink rink@initfour.nl napsal(a):
The PMA_formatNumberByteDown_test test still fails for me on a conversion.... does this pass on other systems?
Six tests do fail for me:
- testEncrypt(PMA_blowfish_test)
- testDecrypt(PMA_blowfish_test)
These have never failed me. Could you look into this?
Unfortunately I lack time to look at them now, here are the failures I got:
1) testEncrypt(PMA_blowfish_test) Failed asserting that two strings are equal. expected string <kO/kc4j/nyk=> difference <xxxxxxxxxxx> got string <Oa6nX6ayG9U=> /home/nijel/work/phpmyadmin/trunk/phpMyAdmin/test/PMA_blowfish_test.php:48
2) testDecrypt(PMA_blowfish_test) Failed asserting that two strings are equal. expected string <12345678> difference <xxxxxxxx> got string <N`��z�> /home/nijel/work/phpmyadmin/trunk/phpMyAdmin/test/PMA_blowfish_test.php:56
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #4
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #5
- testFormatNumber(PMA_formatNumberByteDown_test) with data set #6
These 3 fail me as well, using the data: array(0.00003, 3, 2, '0,03 m'), array(0.003, 3, 3, '0,003 '), array(-0.003, 6, 0, '-3 m'),
Could someone with intimate formatting knowledge take a look at this to determine if PMA_formatNumber() needs fixing, or the testcase?
It looks to me like PMA_formatNumber() is broken (it tends to return zero instead of exact number):
3) testFormatNumber(PMA_formatNumberByteDown_test) with data set #4 (3.0E-5, 3, 2, '0,03 m') Failed asserting that two strings are equal. expected string <0,03 m> difference < xxx> got string <0,00 k> /home/nijel/work/phpmyadmin/trunk/phpMyAdmin/test/PMA_formatNumberByteDown_test.php:86
4) testFormatNumber(PMA_formatNumberByteDown_test) with data set #5 (0.003, 3, 3, '0,003 ') Failed asserting that two strings are equal. expected string <0,003 > difference < xxx> got string <0,000 k> /home/nijel/work/phpmyadmin/trunk/phpMyAdmin/test/PMA_formatNumberByteDown_test.php:86
5) testFormatNumber(PMA_formatNumberByteDown_test) with data set #6 (-0.003, 6, 0, '-3 m') Failed asserting that two strings are equal. expected string <-3 m> difference < xxx> got string <-0 k> /home/nijel/work/phpmyadmin/trunk/phpMyAdmin/test/PMA_formatNumberByteDown_test.php:86
- testWriteReloadNavigation(PMA_headerLocation_test)
Should be fixed now.
Yes it is, thanks!
Herman van Rink a écrit :
Hi,
I've played around with phpunit a bit as of late, and think we could make better use of this for pma.
The PMA_formatNumberByteDown_test test still fails for me on a conversion.... does this pass on other systems?
Fails for me too.
I've setup several new testcases that uses Selenium RC [1] to replay actual browser behaviour. My first test in this is a case where I log in and do some basic browsing and change the test users password. The test scenario's are easily recorded using the Firefox Selenium IDE add-on. [2] Testing can happen in Firefox, Internet Explorer and other browsers.
I've set up a cronjob which would run the test suite e.g. every day on trunk, and plan to expand this to the latest testing branch. Any errors would be mailed to me. If others are interested I could sent this to a list address.
Good initiative, Herman! I was wondering about Selenium but never took the time to try it.
Do you think that the learning curve is appropriate enough, so that we could launch a TestFest, asking groups to build Selenium tests for the project?
Marc Delisle wrote:
Do you think that the learning curve is appropriate enough, so that we could launch a TestFest, asking groups to build Selenium tests for the project?
Getting everything set-up was a bit fiddly, but I'll try to put some setup info in the wiki.
For someone to contribute a testcase the learning curve would be less steep . The Selenium Firefox add-on is all you need to record a testcase. Fine tuning the php code is something an experienced user could take up.
-
Herman van Rink -
Marc Delisle -
Michal Čihař