[Phpmyadmin-devel] Selenium Test Organisation
While studying the current Selenium test cases, I came across a design/organisation issue. This issue is related to redundant actions. For example, if we want to test the creation of a table, we need to create a database. We create a database already in the 'create database' test. Similarly, if go deeper into the flow, to test insertion of records in a table, we need to create a table which in turns needs us to create a database. If we have separate tests for these - (namely, 'Create a table', 'Insert a Record') we will be doing some actions redundantly, that is creating a database and creating a table and so on. Apart from the problem of redundancy, the tests will take a longer time to execute unnecessarily. The alternatives that we might go for are: 1. Create one single test that tests everything in the flow - creating a db -> creating a table -> inserting records, searching, editing (and other tasks) -> drop table -> drop db. The downside is that this will create a very long test in terms of length. 2. Enforce the pre-requisite of having a testdb and some tables in that test db specifically for Selenium test cases wherever Selenium tests are supposed to be run. The downside is that this will create an extra and perhaps unnecessary dependency on the environment. Looking for some inputs on which approach might be better. Thanks! -- Ayush Chaudhary
Hi Ayush, Your pull request [1] about UT print out some output, which make UT failed on php 5.3 https://travis-ci.org/phpmyadmin/phpmyadmin/jobs/9811338 php 5.4 and PHP5.5 is working. check your travil build result: https://travis-ci.org/phpmyadmin/phpmyadmin/builds/9811335 Can you have a check? thanks [1] https://github.com/phpmyadmin/phpmyadmin/pull/553 Bin
Hi all, After two month' coding and sending pull request, I found that we are missing something after sending out pull request. After sending out pull request, we must check that Github shows that "Good to merge — The Travis CI build passed(Details)" but actually, it is *not* good enough to merge, there may be some UT failures, so I think we should also click (Details) [1] to see the details of Travis CI build result for PHP5.5, PHP5.4, PHP5.3 After check that all UT passed, it is Good to merge. Hope it is helpful [1] https://travis-ci.org/phpmyadmin/phpmyadmin/builds/10117870 thanks, Bin
Hi Dne Mon, 12 Aug 2013 23:06:55 +0800 "Bin Zu" <xmujay@gmail.com> napsal(a):
After two month' coding and sending pull request, I found that we are missing something after sending out pull request.
After sending out pull request, we must check that Github shows that "Good to merge — The Travis CI build passed(Details)"
but actually, it is *not* good enough to merge, there may be some UT failures,
What is actually a bug in our ant script, I've fixed it in 0dd6a9a. Now it will correctly fail on unit test failure. -- Michal Čihař | http://cihar.com | http://phpmyadmin.net
Hi Dne Mon, 12 Aug 2013 19:41:47 +0530 Ayush Chaudhary <ayushchd@gmail.com> napsal(a):
While studying the current Selenium test cases, I came across a design/organisation issue. This issue is related to redundant actions. For example, if we want to test the creation of a table, we need to create a database. We create a database already in the 'create database' test. Similarly, if go deeper into the flow, to test insertion of records in a table, we need to create a table which in turns needs us to create a database. If we have separate tests for these - (namely, 'Create a table', 'Insert a Record') we will be doing some actions redundantly, that is creating a database and creating a table and so on. Apart from the problem of redundancy, the tests will take a longer time to execute unnecessarily.
The alternatives that we might go for are:
1. Create one single test that tests everything in the flow - creating a db -> creating a table -> inserting records, searching, editing (and other tasks) -> drop table -> drop db. The downside is that this will create a very long test in terms of length.
2. Enforce the pre-requisite of having a testdb and some tables in that test db specifically for Selenium test cases wherever Selenium tests are supposed to be run. The downside is that this will create an extra and perhaps unnecessary dependency on the environment.
I really did not look deeper into Selenium and PHPUnit integration, but isn't it possible to do some setup before running test, which will create tables used in the test? I'd really prefer to have separate tests for each task (so that it is can be easily seen what is broken), while the environment setup does not have to be done using Selenium... -- Michal Čihař | http://cihar.com | http://phpmyadmin.net
Hi Michal, On Tuesday, August 13, 2013, Michal Čihař wrote:
Hi
Dne Mon, 12 Aug 2013 19:41:47 +0530 Ayush Chaudhary <ayushchd@gmail.com <javascript:;>> napsal(a):
While studying the current Selenium test cases, I came across a design/organisation issue. This issue is related to redundant actions. For example, if we want to test the creation of a table, we need to create a database. We create a database already in the 'create database' test. Similarly, if go deeper into the flow, to test insertion of records in a table, we need to create a table which in turns needs us to create a database. If we have separate tests for these - (namely, 'Create a table', 'Insert a Record') we will be doing some actions redundantly, that is creating a database and creating a table and so on. Apart from the problem of redundancy, the tests will take a longer time to execute unnecessarily.
The alternatives that we might go for are:
1. Create one single test that tests everything in the flow - creating a db -> creating a table -> inserting records, searching, editing (and other tasks) -> drop table -> drop db. The downside is that this will create a very long test in terms of length.
2. Enforce the pre-requisite of having a testdb and some tables in that test db specifically for Selenium test cases wherever Selenium tests are supposed to be run. The downside is that this will create an extra and perhaps unnecessary dependency on the environment.
I really did not look deeper into Selenium and PHPUnit integration, but isn't it possible to do some setup before running test, which will create tables used in the test?
I'd really prefer to have separate tests for each task (so that it is can be easily seen what is broken), while the environment setup does not have to be done using Selenium...
-- Michal Čihař | http://cihar.com | http://phpmyadmin.net
I don't see any explicit provision, however, we could perhaps connect manually to database and execute corresponding SQL queries to configure the database before our tests. Does that sound ok? If yes, we use mysqli?
Hi Dne Tue, 13 Aug 2013 14:07:52 +0530 Ayush Chaudhary <ayushchd@gmail.com> napsal(a):
I don't see any explicit provision, however, we could perhaps connect manually to database and execute corresponding SQL queries to configure the database before our tests. Does that sound ok? If yes, we use mysqli?
There is no option to use setUp method for that? -- Michal Čihař | http://cihar.com | http://phpmyadmin.net
On Tuesday, 13 August 2013 at 2:24 PM, Michal Čihař wrote:
Hi
Dne Tue, 13 Aug 2013 14:07:52 +0530 Ayush Chaudhary <ayushchd@gmail.com (mailto:ayushchd@gmail.com)> napsal(a):
I don't see any explicit provision, however, we could perhaps connect manually to database and execute corresponding SQL queries to configure the database before our tests. Does that sound ok? If yes, we use mysqli?
There is no option to use setUp method for that? These tests run just similar to other unit tests except that we can use Selenium web driver calls using phpunit. So yes, we can use setup method to connect to mysql and create tables, databases that we need for our tests.
-- Michal Èihaø | http://cihar.com | http://phpmyadmin.net
------------------------------------------------------------------------------ Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net (mailto:Phpmyadmin-devel@lists.sourceforge.net) https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
Hi Aysush,
On Tuesday, 13 August 2013 at 2:24 PM, Michal Čihař wrote:
Hi
Dne Tue, 13 Aug 2013 14:07:52 +0530 Ayush Chaudhary <ayushchd@gmail.com (mailto:ayushchd@gmail.com)> napsal(a):
I don't see any explicit provision, however, we could perhaps connect manually to database and execute corresponding SQL queries to configure the database before our tests. Does that sound ok? If yes, we use mysqli?
There is no option to use setUp method for that? These tests run just similar to other unit tests except that we can use Selenium web driver calls using phpunit. So yes, we can use setup method to connect to mysql and create tables, databases that we need for our tests.
That will back to a problem I encountered when I made UT for DBI testing. For selenium case, we should setup database, using username, password to connect. but this can't be deploy easily to CI and other developers' dev box because *the username and password* is private data. please advise! Adam
Hi, On Tuesday, 13 August 2013 at 9:23 PM, adam wrote:
Hi Aysush,
On Tuesday, 13 August 2013 at 2:24 PM, Michal Čihař wrote:
Hi
Dne Tue, 13 Aug 2013 14:07:52 +0530 Ayush Chaudhary <ayushchd@gmail.com (mailto:ayushchd@gmail.com)> napsal(a):
I don't see any explicit provision, however, we could perhaps connect manually to database and execute corresponding SQL queries to configure the database before our tests. Does that sound ok? If yes, we use mysqli?
There is no option to use setUp method for that?
These tests run just similar to other unit tests except that we can use Selenium web driver calls using phpunit. So yes, we can use setup method to connect to mysql and create tables, databases that we need for our tests.
That will back to a problem I encountered when I made UT for DBI testing. For selenium case, we should setup database, using username, password to connect. but this can't be deploy easily to CI and other developers' dev box because *the username and password* is private data. please advise!
True, but if we are to test the functionality of phpMyAdmin, there is no other way but to connect to an actual MySQL Server and execute queries there to setup tests or to verify test results.
Adam ------------------------------------------------------------------------------ Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net (mailto:Phpmyadmin-devel@lists.sourceforge.net) https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
Hi,
Hi,
On Tuesday, 13 August 2013 at 9:23 PM, adam wrote:
Hi Aysush,
On Tuesday, 13 August 2013 at 2:24 PM, Michal Čihař wrote:
Hi
Dne Tue, 13 Aug 2013 14:07:52 +0530 Ayush Chaudhary <ayushchd@gmail.com (mailto:ayushchd@gmail.com)> napsal(a):
I don't see any explicit provision, however, we could perhaps connect manually to database and execute corresponding SQL queries to configure the database before our tests. Does that sound ok? If yes, we use mysqli?
There is no option to use setUp method for that?
These tests run just similar to other unit tests except that we can use Selenium web driver calls using phpunit. So yes, we can use setup method to connect to mysql and create tables, databases that we need for our tests.
That will back to a problem I encountered when I made UT for DBI testing. For selenium case, we should setup database, using username, password to connect. but this can't be deploy easily to CI and other developers' dev box because *the username and password* is private data. please advise!
True, but if we are to test the functionality of phpMyAdmin, there is no other way but to connect to an actual MySQL Server and execute queries there to setup tests or to verify test results.
Agree. If we want to conenct to actual MySQL server, the username and password should be exposed. I saw your pull request about Selenium test case, I will add some cases from my side. thanks, Adam
On Friday, August 16, 2013 at 3:37 AM, adam wrote:
Hi,
Hi,
On Tuesday, 13 August 2013 at 9:23 PM, adam wrote:
Hi Aysush,
On Tuesday, 13 August 2013 at 2:24 PM, Michal Čihař wrote:
Hi Dne Tue, 13 Aug 2013 14:07:52 +0530 Ayush Chaudhary <ayushchd@gmail.com (mailto:ayushchd@gmail.com)> napsal(a):
I don't see any explicit provision, however, we could perhaps connect manually to database and execute corresponding SQL queries to configure the database before our tests. Does that sound ok? If yes, we use mysqli?
There is no option to use setUp method for that?
These tests run just similar to other unit tests except that we can use Selenium web driver calls using phpunit. So yes, we can use setup method to connect to mysql and create tables, databases that we need for our tests.
That will back to a problem I encountered when I made UT for DBI testing. For selenium case, we should setup database, using username, password to connect. but this can't be deploy easily to CI and other developers' dev box because *the username and password* is private data. please advise!
True, but if we are to test the functionality of phpMyAdmin, there is no other way but to connect to an actual MySQL Server and execute queries there to setup tests or to verify test results.
Agree. If we want to conenct to actual MySQL server, the username and password should be exposed. I saw your pull request about Selenium test case, I will add some cases from my side.
thanks, Adam
Cant we use environment variables or doesnt php have access to those. When I needed sensitive data I either stored them in environment variables or in file not tracked by the repo that I include. In both of those situations custom login credentials can be set up on every installation of phpmyadmin where the tests should run. The tests can just check for the existance of the file or variable otherwise disable the part of the test suite that depends on it.
------------------------------------------------------------------------------ Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
participants (5)
-
adam -
Ayush Chaudhary -
Bin Zu -
Michal Čihař -
Mohamed Ashraf