Hi
You again missed reply to list, moving discussion back there :-).
Dne Tue, 12 Jul 2011 18:48:57 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
On Tue, Jul 12, 2011 at 3:49 PM, Michal Čihař michal@cihar.com wrote:
Hi
Dne Tue, 12 Jul 2011 13:08:37 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
And one more suggestion:
Counting rows in big tables that use CSV as Engine (such as the general_log) seem very slow. Maybe rows should not be counted automatically for CSV Tables and only done upon user request. In my test I counted 36k rows on the demo server, that takes around 250ms, so imagine the general_log running all day. Then you will have 1mil+ rows, which then requires ~6-8 seconds to count.
There is already similar logic for InnoDB or views, so only another engine should be added here.
Do you know in which file/line this is?
It should be libraries/display_tbl.lib.php somewhere near usage of MaxExactCount.
Next suggestion :D Just saw in header_scripts.inc.php that codemirror.js and mysql.js is included globally. Isn't that a bit overkill? Not every page (like my status page) requires codemirror.
I don't think it hurts that much (browser should have that file in cache), generally I was too lazy to figure out in which all places SQL box needing highlighting might appear.
2011/7/12 Michal Čihař michal@cihar.com:
Hi
You again missed reply to list, moving discussion back there :-).
Dne Tue, 12 Jul 2011 18:48:57 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
On Tue, Jul 12, 2011 at 3:49 PM, Michal Čihař michal@cihar.com wrote:
Hi
Dne Tue, 12 Jul 2011 13:08:37 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
And one more suggestion:
Counting rows in big tables that use CSV as Engine (such as the general_log) seem very slow. Maybe rows should not be counted automatically for CSV Tables and only done upon user request. In my test I counted 36k rows on the demo server, that takes around 250ms, so imagine the general_log running all day. Then you will have 1mil+ rows, which then requires ~6-8 seconds to count.
There is already similar logic for InnoDB or views, so only another engine should be added here.
Do you know in which file/line this is?
It should be libraries/display_tbl.lib.php somewhere near usage of MaxExactCount.
This code there is seriously odd. $unlim_num_rows seems to be the total count of rows, which, from what I can see, is calculated in PMA_setDisplayMode() There it calls PMA_Table::countRecords($db, $table) without checking for views or innodb. countRecords() in Table.class.php I can see it doesn't calculate the count for views or only up to a limit of MaxExactCountViews.
But I don't see any limit being applied when the table engine is InnoDB.
Either way, for limiting the count on CSV Engine tables, I guess that should be done in countRecords() in Table.class.php?
Next suggestion :D Just saw in header_scripts.inc.php that codemirror.js and mysql.js is included globally. Isn't that a bit overkill? Not every page (like my status page) requires codemirror.
I don't think it hurts that much (browser should have that file in cache), generally I was too lazy to figure out in which all places SQL box needing highlighting might appear.
But the javascript still needs to be parsed and executed by the browser each time, adding delay. Also it still requires it's own GET request, returning 304. Chrome seems to even skip the GET requests if the page is not refreshed, but firefox 5 doesn't do that in my tests.
The status page currently loads 21 seperate javascript files, and the delay introduced by this gets quite the noticeable. So I would like to keep the amount of loaded js files as small as possible by removing what is not required - codemiror.js + mysql.js should only go where it's needed (-2) - load chart export on demand (-3) - load monitor js code on demand (-2.5) - We could merge always included files into one: functions.js, jquery, jquery.ui, jquery.qtip (-3)
Then the amount of loaded files would be half already, in the case of the status page.
Checking with chromes devtools it takes around 4 seconds to load the status page. With js disabled it takes around 2.3 seconds. So the js accounts for almost half the loading time (and CSS Sprites could probably save us half of that 2.3secs).
With the js files reduced, the not immediately needed js code loaded only on demand and css sprites I bet we could get the loading time down to 2 seconds. 4s is way to much imo.
-- Michal Čihař | http://cihar.com | http://phpmyadmin.cz
AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on "Lean Startup Secrets Revealed." This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
On Thu, 2011-07-14 at 11:14 +0300, Tyron Madlener wrote:
2011/7/12 Michal Čihař michal@cihar.com:
Hi
You again missed reply to list, moving discussion back there :-).
Dne Tue, 12 Jul 2011 18:48:57 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
On Tue, Jul 12, 2011 at 3:49 PM, Michal Čihař michal@cihar.com wrote:
Hi
Dne Tue, 12 Jul 2011 13:08:37 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
And one more suggestion:
Counting rows in big tables that use CSV as Engine (such as the general_log) seem very slow. Maybe rows should not be counted automatically for CSV Tables and only done upon user request. In my test I counted 36k rows on the demo server, that takes around 250ms, so imagine the general_log running all day. Then you will have 1mil+ rows, which then requires ~6-8 seconds to count.
There is already similar logic for InnoDB or views, so only another engine should be added here.
Do you know in which file/line this is?
It should be libraries/display_tbl.lib.php somewhere near usage of MaxExactCount.
This code there is seriously odd. $unlim_num_rows seems to be the total count of rows, which, from what I can see, is calculated in PMA_setDisplayMode() There it calls PMA_Table::countRecords($db, $table) without checking for views or innodb. countRecords() in Table.class.php I can see it doesn't calculate the count for views or only up to a limit of MaxExactCountViews.
But I don't see any limit being applied when the table engine is InnoDB.
Either way, for limiting the count on CSV Engine tables, I guess that should be done in countRecords() in Table.class.php?
Next suggestion :D Just saw in header_scripts.inc.php that codemirror.js and mysql.js is included globally. Isn't that a bit overkill? Not every page (like my status page) requires codemirror.
I don't think it hurts that much (browser should have that file in cache), generally I was too lazy to figure out in which all places SQL box needing highlighting might appear.
But the javascript still needs to be parsed and executed by the browser each time, adding delay. Also it still requires it's own GET request, returning 304. Chrome seems to even skip the GET requests if the page is not refreshed, but firefox 5 doesn't do that in my tests.
The status page currently loads 21 seperate javascript files, and the delay introduced by this gets quite the noticeable. So I would like to keep the amount of loaded js files as small as possible by removing what is not required
- codemiror.js + mysql.js should only go where it's needed (-2)
- load chart export on demand (-3)
- load monitor js code on demand (-2.5)
- We could merge always included files into one: functions.js, jquery,
jquery.ui, jquery.qtip (-3)
No way. Merging jquery with functions.js may save the users a few milliseconds per page load, but will add hours of hacking to the developers!
Then the amount of loaded files would be half already, in the case of the status page.
Checking with chromes devtools it takes around 4 seconds to load the status page. With js disabled it takes around 2.3 seconds. So the js accounts for almost half the loading time (and CSS Sprites could probably save us half of that 2.3secs).
With the js files reduced, the not immediately needed js code loaded only on demand and css sprites I bet we could get the loading time down to 2 seconds. 4s is way to much imo.
On Thu, Jul 14, 2011 at 1:28 PM, Rouslan Placella rouslan@placella.com wrote:
On Thu, 2011-07-14 at 11:14 +0300, Tyron Madlener wrote:
2011/7/12 Michal Čihař michal@cihar.com:
Hi
You again missed reply to list, moving discussion back there :-).
Dne Tue, 12 Jul 2011 18:48:57 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
On Tue, Jul 12, 2011 at 3:49 PM, Michal Čihař michal@cihar.com wrote:
Hi
Dne Tue, 12 Jul 2011 13:08:37 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
And one more suggestion:
Counting rows in big tables that use CSV as Engine (such as the general_log) seem very slow. Maybe rows should not be counted automatically for CSV Tables and only done upon user request. In my test I counted 36k rows on the demo server, that takes around 250ms, so imagine the general_log running all day. Then you will have 1mil+ rows, which then requires ~6-8 seconds to count.
There is already similar logic for InnoDB or views, so only another engine should be added here.
Do you know in which file/line this is?
It should be libraries/display_tbl.lib.php somewhere near usage of MaxExactCount.
This code there is seriously odd. $unlim_num_rows seems to be the total count of rows, which, from what I can see, is calculated in PMA_setDisplayMode() There it calls PMA_Table::countRecords($db, $table) without checking for views or innodb. countRecords() in Table.class.php I can see it doesn't calculate the count for views or only up to a limit of MaxExactCountViews.
But I don't see any limit being applied when the table engine is InnoDB.
Either way, for limiting the count on CSV Engine tables, I guess that should be done in countRecords() in Table.class.php?
Next suggestion :D Just saw in header_scripts.inc.php that codemirror.js and mysql.js is included globally. Isn't that a bit overkill? Not every page (like my status page) requires codemirror.
I don't think it hurts that much (browser should have that file in cache), generally I was too lazy to figure out in which all places SQL box needing highlighting might appear.
But the javascript still needs to be parsed and executed by the browser each time, adding delay. Also it still requires it's own GET request, returning 304. Chrome seems to even skip the GET requests if the page is not refreshed, but firefox 5 doesn't do that in my tests.
The status page currently loads 21 seperate javascript files, and the delay introduced by this gets quite the noticeable. So I would like to keep the amount of loaded js files as small as possible by removing what is not required
- codemiror.js + mysql.js should only go where it's needed (-2)
- load chart export on demand (-3)
- load monitor js code on demand (-2.5)
- We could merge always included files into one: functions.js, jquery,
jquery.ui, jquery.qtip (-3)
No way. Merging jquery with functions.js may save the users a few milliseconds per page load, but will add hours of hacking to the developers!
Doesn't have to be in one file literally. It's also possible to build a php code snippet that packs the files together and sends it to the browser. Define a development mode where it doesn't do that.
Then the amount of loaded files would be half already, in the case of the status page.
Checking with chromes devtools it takes around 4 seconds to load the status page. With js disabled it takes around 2.3 seconds. So the js accounts for almost half the loading time (and CSS Sprites could probably save us half of that 2.3secs).
With the js files reduced, the not immediately needed js code loaded only on demand and css sprites I bet we could get the loading time down to 2 seconds. 4s is way to much imo.
AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on "Lean Startup Secrets Revealed." This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
On Thu, 2011-07-14 at 13:37 +0300, Tyron Madlener wrote:
On Thu, Jul 14, 2011 at 1:28 PM, Rouslan Placella rouslan@placella.com wrote:
On Thu, 2011-07-14 at 11:14 +0300, Tyron Madlener wrote:
2011/7/12 Michal Čihař michal@cihar.com:
Hi
You again missed reply to list, moving discussion back there :-).
Dne Tue, 12 Jul 2011 18:48:57 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
On Tue, Jul 12, 2011 at 3:49 PM, Michal Čihař michal@cihar.com wrote:
Hi
Dne Tue, 12 Jul 2011 13:08:37 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
> And one more suggestion: > > Counting rows in big tables that use CSV as Engine (such as the > general_log) seem very slow. Maybe rows should not be counted > automatically for CSV Tables and only done upon user request. > In my test I counted 36k rows on the demo server, that takes around > 250ms, so imagine the general_log running all day. Then you will have > 1mil+ rows, which then requires ~6-8 seconds to count.
There is already similar logic for InnoDB or views, so only another engine should be added here.
Do you know in which file/line this is?
It should be libraries/display_tbl.lib.php somewhere near usage of MaxExactCount.
This code there is seriously odd. $unlim_num_rows seems to be the total count of rows, which, from what I can see, is calculated in PMA_setDisplayMode() There it calls PMA_Table::countRecords($db, $table) without checking for views or innodb. countRecords() in Table.class.php I can see it doesn't calculate the count for views or only up to a limit of MaxExactCountViews.
But I don't see any limit being applied when the table engine is InnoDB.
Either way, for limiting the count on CSV Engine tables, I guess that should be done in countRecords() in Table.class.php?
Next suggestion :D Just saw in header_scripts.inc.php that codemirror.js and mysql.js is included globally. Isn't that a bit overkill? Not every page (like my status page) requires codemirror.
I don't think it hurts that much (browser should have that file in cache), generally I was too lazy to figure out in which all places SQL box needing highlighting might appear.
But the javascript still needs to be parsed and executed by the browser each time, adding delay. Also it still requires it's own GET request, returning 304. Chrome seems to even skip the GET requests if the page is not refreshed, but firefox 5 doesn't do that in my tests.
The status page currently loads 21 seperate javascript files, and the delay introduced by this gets quite the noticeable. So I would like to keep the amount of loaded js files as small as possible by removing what is not required
- codemiror.js + mysql.js should only go where it's needed (-2)
- load chart export on demand (-3)
- load monitor js code on demand (-2.5)
- We could merge always included files into one: functions.js, jquery,
jquery.ui, jquery.qtip (-3)
No way. Merging jquery with functions.js may save the users a few milliseconds per page load, but will add hours of hacking to the developers!
Doesn't have to be in one file literally. It's also possible to build a php code snippet that packs the files together and sends it to the browser. Define a development mode where it doesn't do that.
I think that it would still interfere with debugging and the benefits would be marginal, only 2 less HTTP requests.
That said, I'm all up for the conditional includes of the other files you mention :)
Then the amount of loaded files would be half already, in the case of the status page.
Checking with chromes devtools it takes around 4 seconds to load the status page. With js disabled it takes around 2.3 seconds. So the js accounts for almost half the loading time (and CSS Sprites could probably save us half of that 2.3secs).
With the js files reduced, the not immediately needed js code loaded only on demand and css sprites I bet we could get the loading time down to 2 seconds. 4s is way to much imo.
AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on "Lean Startup Secrets Revealed." This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on "Lean Startup Secrets Revealed." This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
On Thu, Jul 14, 2011 at 1:42 PM, Rouslan Placella rouslan@placella.com wrote:
On Thu, 2011-07-14 at 13:37 +0300, Tyron Madlener wrote:
On Thu, Jul 14, 2011 at 1:28 PM, Rouslan Placella rouslan@placella.com wrote:
On Thu, 2011-07-14 at 11:14 +0300, Tyron Madlener wrote:
2011/7/12 Michal Čihař michal@cihar.com:
Hi
You again missed reply to list, moving discussion back there :-).
Dne Tue, 12 Jul 2011 18:48:57 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
On Tue, Jul 12, 2011 at 3:49 PM, Michal Čihař michal@cihar.com wrote: > Hi > > Dne Tue, 12 Jul 2011 13:08:37 +0300 > Tyron Madlener tyronx@gmail.com napsal(a): > >> And one more suggestion: >> >> Counting rows in big tables that use CSV as Engine (such as the >> general_log) seem very slow. Maybe rows should not be counted >> automatically for CSV Tables and only done upon user request. >> In my test I counted 36k rows on the demo server, that takes around >> 250ms, so imagine the general_log running all day. Then you will have >> 1mil+ rows, which then requires ~6-8 seconds to count. > > There is already similar logic for InnoDB or views, so only another > engine should be added here.
Do you know in which file/line this is?
It should be libraries/display_tbl.lib.php somewhere near usage of MaxExactCount.
This code there is seriously odd. $unlim_num_rows seems to be the total count of rows, which, from what I can see, is calculated in PMA_setDisplayMode() There it calls PMA_Table::countRecords($db, $table) without checking for views or innodb. countRecords() in Table.class.php I can see it doesn't calculate the count for views or only up to a limit of MaxExactCountViews.
But I don't see any limit being applied when the table engine is InnoDB.
Either way, for limiting the count on CSV Engine tables, I guess that should be done in countRecords() in Table.class.php?
Next suggestion :D Just saw in header_scripts.inc.php that codemirror.js and mysql.js is included globally. Isn't that a bit overkill? Not every page (like my status page) requires codemirror.
I don't think it hurts that much (browser should have that file in cache), generally I was too lazy to figure out in which all places SQL box needing highlighting might appear.
But the javascript still needs to be parsed and executed by the browser each time, adding delay. Also it still requires it's own GET request, returning 304. Chrome seems to even skip the GET requests if the page is not refreshed, but firefox 5 doesn't do that in my tests.
The status page currently loads 21 seperate javascript files, and the delay introduced by this gets quite the noticeable. So I would like to keep the amount of loaded js files as small as possible by removing what is not required
- codemiror.js + mysql.js should only go where it's needed (-2)
- load chart export on demand (-3)
- load monitor js code on demand (-2.5)
- We could merge always included files into one: functions.js, jquery,
jquery.ui, jquery.qtip (-3)
No way. Merging jquery with functions.js may save the users a few milliseconds per page load, but will add hours of hacking to the developers!
Doesn't have to be in one file literally. It's also possible to build a php code snippet that packs the files together and sends it to the browser. Define a development mode where it doesn't do that.
I think that it would still interfere with debugging and the benefits would be marginal, only 2 less HTTP requests.
That said, I'm all up for the conditional includes of the other files you mention :)
Okay. For now I am going to implement css sprites.
Then the amount of loaded files would be half already, in the case of the status page.
Checking with chromes devtools it takes around 4 seconds to load the status page. With js disabled it takes around 2.3 seconds. So the js accounts for almost half the loading time (and CSS Sprites could probably save us half of that 2.3secs).
With the js files reduced, the not immediately needed js code loaded only on demand and css sprites I bet we could get the loading time down to 2 seconds. 4s is way to much imo.
AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on "Lean Startup Secrets Revealed." This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on "Lean Startup Secrets Revealed." This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on "Lean Startup Secrets Revealed." This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev _______________________________________________ Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel
Hi
Dne Thu, 14 Jul 2011 11:14:26 +0300 Tyron Madlener tyronx@gmail.com napsal(a):
2011/7/12 Michal Čihař michal@cihar.com:
It should be libraries/display_tbl.lib.php somewhere near usage of MaxExactCount.
This code there is seriously odd. $unlim_num_rows seems to be the total count of rows, which, from what I can see, is calculated in PMA_setDisplayMode() There it calls PMA_Table::countRecords($db, $table) without checking for views or innodb. countRecords() in Table.class.php I can see it doesn't calculate the count for views or only up to a limit of MaxExactCountViews.
But I don't see any limit being applied when the table engine is InnoDB.
Either way, for limiting the count on CSV Engine tables, I guess that should be done in countRecords() in Table.class.php?
It should be done for both in same places and using same logic. If it currently does not work for InnoDB that needs to be fixed as well.
The status page currently loads 21 seperate javascript files, and the delay introduced by this gets quite the noticeable. So I would like to keep the amount of loaded js files as small as possible by removing what is not required
- codemiror.js + mysql.js should only go where it's needed (-2)
I've included these ones on places where they should be needed, still might need some adjustments for AJAX (though it's not dynamically attached anywhere, so I don't think there will be a problem).
- load chart export on demand (-3)
- load monitor js code on demand (-2.5)
- We could merge always included files into one: functions.js, jquery,
jquery.ui, jquery.qtip (-3)
Merging third party scripts will cause problems on upgrading them, but maybe this is something what could be done on release time (similar to compression of javascript we do on release).
Then the amount of loaded files would be half already, in the case of the status page.
Checking with chromes devtools it takes around 4 seconds to load the status page. With js disabled it takes around 2.3 seconds. So the js accounts for almost half the loading time (and CSS Sprites could probably save us half of that 2.3secs).
With the js files reduced, the not immediately needed js code loaded only on demand and css sprites I bet we could get the loading time down to 2 seconds. 4s is way to much imo.
4s sounds definitely too much.
-
Michal Čihař -
Rouslan Placella -
Tyron Madlener