Hi all (and especially Yasitha)
I've merged some new test cases, but we're getting more and more failures:
http://ci.phpmyadmin.net/job/phpMyAdmin-continuous/lastCompletedBuild/testRe...
Most of them being caused by not properly set up environment - they silently expect some parameters are unset while they are set by other testcase (usually things like server, db or token).
Hi all (and especially Yasitha)
I've merged some new test cases, but we're getting more and more failures:
http://ci.phpmyadmin.net/job/phpMyAdmin-continuous/lastCompletedBuild/testRe...
Most of them being caused by not properly set up environment - they silently expect some parameters are unset while they are set by other testcase (usually things like server, db or token).
Hi Michal,
The tests failures may because of my changes regarding introduce private properties. I were trying to fix those, but not fixed yet.
Yesterday I couldn't involve with either fixing those test failures or do the task you have assigned me. Because I found some bugs in my code, I fix them yesterday itself. I'm intending to involve with improve tests after completing that task.
Since my changes have been committed to the master, if possible Yasitha also can join with me to improve the tests.
Regards !
Hi
Dne Tue, 10 Jul 2012 13:28:29 +0530 Chanaka Dharmarathna pe.chanaka.ck@gmail.com napsal(a):
The tests failures may because of my changes regarding introduce private properties. I were trying to fix those, but not fixed yet.
Well you're responsible for some of breakages, but definitely not majority of them :-). Actually I was not about failing individual tests, but rather on tests which can be run separately, but fail when whole testsuite is being executed.
Many of the tests expect they will get a clean environment and rely on some parameters not being set. This is not true when whole testsuite runs as lot of code still relies on global variables. One of example are parameters being added to URL. Possible fix is following:
https://github.com/phpmyadmin/phpmyadmin/commit/0f32f805d49166c2ba9f33323a54...
Hi Michal,
Got your point.Those test cases are passing when they run along. But can fail when run all the tests. Some global parameters are set at the setup method as well as in side the test method. Have to unset them after doing assertion, I am working on this now.
Regards,
On Tue, Jul 10, 2012 at 3:10 PM, yasitha pandithawatta yasitha4@gmail.comwrote:
Hi Michal,
Got your point.Those test cases are passing when they run along. But can fail when run all the tests. Some global parameters are set at the setup method as well as in side the test method. Have to unset them after doing assertion, I am working on this now.
Regards,
Yasitha Pandithawatta Undergraduate Computer Science and Engineering Department University of Moratuwa Sri Lanka
Hi Michal,
I am working on test failures you pointed out. I have fixed some of them. Most of the failures due to PHP_Invoker_TimeoutException. Is Jenkins test suit run in strict mode? Those test cases are passing when the test case run lonely. What will be the problem for those failures. I couldn't run the whole test. It seems some test cases are waiting for some event. I tried phpunit --strict also but didn't work. Should I spent time at this moment to fix those or continuing on remaining libraries?
Regards,
Hi
Dne Wed, 11 Jul 2012 18:04:21 +0530 yasitha pandithawatta yasitha4@gmail.com napsal(a):
I am working on test failures you pointed out. I have fixed some of them. Most of the failures due to PHP_Invoker_TimeoutException.
If some tests can take longer, they should be tagged @group medium or @group large (this is already true for some tests).
Is Jenkins test suit run in strict mode?
Yes it is using phpunit.xml.dist
Those test cases are passing when the test case run lonely. What will be the problem for those failures. I couldn't run the whole test.
Why you can not run it? Running same tests as on ci.phpmyadmin.net can be done simply by phpunit --configuration phpunit.xml.dist in top level directory.
On Wed, Jul 11, 2012 at 6:41 PM, Michal Čihař michal@cihar.com wrote:
Hi
Dne Wed, 11 Jul 2012 18:04:21 +0530 yasitha pandithawatta yasitha4@gmail.com napsal(a):
I am working on test failures you pointed out. I have fixed some of them. Most of the failures due to PHP_Invoker_TimeoutException.
If some tests can take longer, they should be tagged @group medium or @group large (this is already true for some tests).
Is Jenkins test suit run in strict mode?
Yes it is using phpunit.xml.dist
Those test cases are passing when the test case run lonely. What will be the problem for those failures. I couldn't run
the
whole test.
Why you can not run it? Running same tests as on ci.phpmyadmin.net can be done simply by phpunit --configuration phpunit.xml.dist in top level directory.
Hi Michal,
That is the way I am using too. But after 10 tests it keep waiting for long time. Nothing happens. So I used to run files by files. That may the case I missed those failures.
[image: Inline image 1]
Hi
Dne Wed, 11 Jul 2012 18:50:45 +0530 yasitha pandithawatta yasitha4@gmail.com napsal(a):
That is the way I am using too. But after 10 tests it keep waiting for long time. Nothing happens. So I used to run files by files. That may the case I missed those failures.
Well yes, the test takes too long this way (it takes about 20 minutes on ci.phpmyadmin.net). I have no clue why this happens, but it did not happen few months ago (I think it started with some phpunit update, AFAIR removing php-xdebug helped, but that's not a solution). In case you would be able to figure out why does this happen, it would be great. Otherwise we have to live with too long running testsuite.
On Wed, Jul 11, 2012 at 7:18 PM, Michal Čihař michal@cihar.com wrote:
Hi
Dne Wed, 11 Jul 2012 18:50:45 +0530 yasitha pandithawatta yasitha4@gmail.com napsal(a):
That is the way I am using too. But after 10 tests it keep waiting for
long
time. Nothing happens. So I used to run files by files. That may the case I missed those
failures.
Well yes, the test takes too long this way (it takes about 20 minutes on ci.phpmyadmin.net). I have no clue why this happens, but it did not happen few months ago (I think it started with some phpunit update, AFAIR removing php-xdebug helped, but that's not a solution). In case you would be able to figure out why does this happen, it would be great. Otherwise we have to live with too long running testsuite.
-- Michal Čihař | http://cihar.com | http://phpmyadmin.cz
Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
Hi Michal,
Thanks for the reply. I kept it around 20 minutes and finally got it run. I didn't wait much time earlier since normally it won't take much time. Problem is with only few test cases. If we can identify those test cases, probably can fix it. I'll try to find it. It is better to fix it because then everybody can run whole test suit before committing. This will reduce the test failures caused by day to day commits. So we can get the real use of testing.
Regards,
On Wed, Jul 11, 2012 at 8:09 PM, yasitha pandithawatta yasitha4@gmail.comwrote:
On Wed, Jul 11, 2012 at 7:18 PM, Michal Čihař michal@cihar.com wrote:
Hi
Dne Wed, 11 Jul 2012 18:50:45 +0530 yasitha pandithawatta yasitha4@gmail.com napsal(a):
That is the way I am using too. But after 10 tests it keep waiting for
long
time. Nothing happens. So I used to run files by files. That may the case I missed those
failures.
Well yes, the test takes too long this way (it takes about 20 minutes on ci.phpmyadmin.net). I have no clue why this happens, but it did not happen few months ago (I think it started with some phpunit update, AFAIR removing php-xdebug helped, but that's not a solution). In case you would be able to figure out why does this happen, it would be great. Otherwise we have to live with too long running testsuite.
-- Michal Čihař | http://cihar.com | http://phpmyadmin.cz
Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
Hi Michal,
Thanks for the reply. I kept it around 20 minutes and finally got it run. I didn't wait much time earlier since normally it won't take much time. Problem is with only few test cases. If we can identify those test cases, probably can fix it. I'll try to find it. It is better to fix it because then everybody can run whole test suit before committing. This will reduce the test failures caused by day to day commits. So we can get the real use of testing.
Regards,
Yasitha Pandithawatta Undergraduate Computer Science and Engineering Department University of Moratuwa Sri Lanka
Hi Michal,
I found the reason for the test takes too much time. It because of ob_start() method. This method is calling in two place in whole test suit.
1. testAddRule in PMA_Advisor_test.php with data set #6. There it calls the ruleExprEvaluate() and it has ob_start() call.
*ob_start();* * eval('$value = '.$expr.';');* * $err = ob_get_contents();* * ob_end_clean();* * * 2. start() method in OutputBuffering.class.php. It has the same method call. This method calls when method PMA_fatalError calls. It includes the ./libraries/error.inc.php file and it initiate the PMA_Response object. OutputBuffering is initiate and start method is called in the PMA_Response construct. PMA_fatalError_test.php and PMA_warnMissingExtension_test.php test classes calls the PMA_fatalError method.
I found the exact place by debugging and running whole test case. When those test cases are commented it takes only less than 1 minute to run the whole test suit. :-)
Still I don't have an idea why this happens and it says ob_start() is having some sort of performance issues. Any idea on how fix this?
BTW I have submitted my mid evaluation form.
Regards,
Hi
Dne Thu, 12 Jul 2012 23:20:17 +0530 yasitha pandithawatta yasitha4@gmail.com napsal(a):
I found the exact place by debugging and running whole test case. When those test cases are commented it takes only less than 1 minute to run the whole test suit. :-)
Still I don't have an idea why this happens and it says ob_start() is having some sort of performance issues. Any idea on how fix this?
Thanks for the analysis! I've looked deeper into it and the problem seems to be caused by error inside ob_start/ob_end_clean. Once you remove last test for Advisor class, it get's faster to other tests. I assume in second case the cause is same.
Still I have no clue how to debug furher, maybe somebody else has an idea?
Le 2012-07-12 16:26, Michal Čihař a écrit :
Hi
Dne Thu, 12 Jul 2012 23:20:17 +0530 yasitha pandithawatta yasitha4@gmail.com napsal(a):
I found the exact place by debugging and running whole test case. When those test cases are commented it takes only less than 1 minute to run the whole test suit. :-)
Still I don't have an idea why this happens and it says ob_start() is having some sort of performance issues. Any idea on how fix this?
Thanks for the analysis! I've looked deeper into it and the problem seems to be caused by error inside ob_start/ob_end_clean. Once you remove last test for Advisor class, it get's faster to other tests. I assume in second case the cause is same.
Still I have no clue how to debug furher, maybe somebody else has an idea?
Sorry, are you talking about just running phpunit or all the tests in Jenkins? Because on my test server, running phpunit with 971 tests takes 9 seconds (phpunit 3.6.10, PHP 5.3.13).
Hi
Dne Fri, 13 Jul 2012 08:09:13 -0400 Marc Delisle marc@infomarc.info napsal(a):
Sorry, are you talking about just running phpunit or all the tests in Jenkins? Because on my test server, running phpunit with 971 tests takes 9 seconds (phpunit 3.6.10, PHP 5.3.13).
About whole testsuite, it happens to me on my notebook as well.
Do you have xdebug extension installed (for coverage reports)?
Without xdebug the tests take 15 seconds here, with xdebug something more than 20 minutes (what gets reduced when disabling output buffering as described by Yasitha).
I'm using PHP 5.4.4, xdebug 2.2.0 and phpunit 3.6.10.
Le 2012-07-13 08:40, Michal Čihař a écrit :
Hi
Dne Fri, 13 Jul 2012 08:09:13 -0400 Marc Delisle marc@infomarc.info napsal(a):
Sorry, are you talking about just running phpunit or all the tests in Jenkins? Because on my test server, running phpunit with 971 tests takes 9 seconds (phpunit 3.6.10, PHP 5.3.13).
About whole testsuite, it happens to me on my notebook as well.
Do you have xdebug extension installed (for coverage reports)?
Nope, but I'll install it and report later.
Without xdebug the tests take 15 seconds here, with xdebug something more than 20 minutes (what gets reduced when disabling output buffering as described by Yasitha).
I'm using PHP 5.4.4, xdebug 2.2.0 and phpunit 3.6.10.
Le 2012-07-13 08:49, Marc Delisle a écrit :
Le 2012-07-13 08:40, Michal Čihař a écrit :
Hi
Dne Fri, 13 Jul 2012 08:09:13 -0400 Marc Delisle marc@infomarc.info napsal(a):
Sorry, are you talking about just running phpunit or all the tests in Jenkins? Because on my test server, running phpunit with 971 tests takes 9 seconds (phpunit 3.6.10, PHP 5.3.13).
About whole testsuite, it happens to me on my notebook as well.
Do you have xdebug extension installed (for coverage reports)?
Nope, but I'll install it and report later.
I tried with PHP 5.3.10, xdebug 2.2.0 and phpunit 3.6.10 but I get a memory allocation error when running the tests.
Without xdebug the tests take 15 seconds here, with xdebug something more than 20 minutes (what gets reduced when disabling output buffering as described by Yasitha).
I'm using PHP 5.4.4, xdebug 2.2.0 and phpunit 3.6.10.
Le 2012-07-13 11:22, Marc Delisle a écrit :
Le 2012-07-13 08:49, Marc Delisle a écrit :
Le 2012-07-13 08:40, Michal Čihař a écrit :
Hi
Dne Fri, 13 Jul 2012 08:09:13 -0400 Marc Delisle marc@infomarc.info napsal(a):
Sorry, are you talking about just running phpunit or all the tests in Jenkins? Because on my test server, running phpunit with 971 tests takes 9 seconds (phpunit 3.6.10, PHP 5.3.13).
About whole testsuite, it happens to me on my notebook as well.
Do you have xdebug extension installed (for coverage reports)?
Nope, but I'll install it and report later.
I tried with PHP 5.3.10, xdebug 2.2.0 and phpunit 3.6.10 but I get a memory allocation error when running the tests.
... and the same with PHP 5.4.4, xdebug 2.2.0 and phpunix 3.6.10.
Without xdebug the tests take 15 seconds here, with xdebug something more than 20 minutes (what gets reduced when disabling output buffering as described by Yasitha).
I'm using PHP 5.4.4, xdebug 2.2.0 and phpunit 3.6.10.
On Sat, Jul 14, 2012 at 7:18 PM, Marc Delisle marc@infomarc.info wrote:
Le 2012-07-13 11:22, Marc Delisle a écrit :
Le 2012-07-13 08:49, Marc Delisle a écrit :
Le 2012-07-13 08:40, Michal Čihař a écrit :
Hi
Dne Fri, 13 Jul 2012 08:09:13 -0400 Marc Delisle marc@infomarc.info napsal(a):
Sorry, are you talking about just running phpunit or all the tests in Jenkins? Because on my test server, running phpunit with 971 tests
takes
9 seconds (phpunit 3.6.10, PHP 5.3.13).
About whole testsuite, it happens to me on my notebook as well.
Do you have xdebug extension installed (for coverage reports)?
Nope, but I'll install it and report later.
I tried with PHP 5.3.10, xdebug 2.2.0 and phpunit 3.6.10 but I get a memory allocation error when running the tests.
... and the same with PHP 5.4.4, xdebug 2.2.0 and phpunix 3.6.10.
Without xdebug the tests take 15 seconds here, with xdebug something more than 20 minutes (what gets reduced when disabling output buffering as described by Yasitha).
I'm using PHP 5.4.4, xdebug 2.2.0 and phpunit 3.6.10.
-- Marc Delisle http://infomarc.info
Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
Hi Michal,
Any way it is better to fix it. I am currently working on other failing test cases.
Regards,
-
Chanaka Dharmarathna -
Marc Delisle -
Michal Čihař -
yasitha pandithawatta