Hugues Peccatte a écrit :
2014-10-19 21:53 GMT+02:00 Marc Delisle marc@infomarc.info:
Le 2014-10-19 15:01, Hugues Peccatte a écrit :
2014-10-19 13:21 GMT+02:00 Marc Delisle <marc@infomarc.info mailto:marc@infomarc.info>:
Le 2014-10-18 14:48, Hugues Peccatte a écrit : > 2014-10-13 12:21 GMT+02:00 Hugues Peccatte <hugues.peccatte@gmail.com mailto:hugues.peccatte@gmail.com
> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>>: > > 2014-10-12 21:06 GMT+02:00 Marc Delisle <marc@infomarc.info> <mailto:marc@infomarc.info <mailto:marc@infomarc.info>>>: > > Le 2014-10-12 12:57, Hugues Peccatte a écrit : > > 2014-10-05 20:36 GMT+02:00 Hugues Peccatte <hugues.peccatte@gmail.com mailto:hugues.peccatte@gmail.com
<mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com > > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>>>:
> > > > 2014-10-04 9:01 GMT+02:00 Hugues Peccatte <hugues.peccatte@gmail.com mailto:hugues.peccatte@gmail.com
<mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com > > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>>>:
> > > > Le 4 oct. 2014 03:22, "Madhura Jayaratne" <madhura.cj@gmail.com mailto:madhura.cj@gmail.com
<mailto:madhura.cj@gmail.com <mailto:madhura.cj@gmail.com>> > > <mailto:madhura.cj@gmail.com <mailto:madhura.cj@gmail.com> > <mailto:madhura.cj@gmail.com <mailto:madhura.cj@gmail.com>>>>a écrit :
> > > > > > > > > > > > > > > > On Sat, Oct 4, 2014 at 1:24 AM, Hugues Peccatte > > <hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>
> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>
> <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com> > <mailto:hugues.peccatte@gmail.com <mailto:hugues.peccatte@gmail.com>>>>
> > wrote: > > >> > > >> 2014-10-03 12:26 GMT+02:00 Marc Delisle <marc@infomarc.info mailto:marc@infomarc.info
<mailto:marc@infomarc.info <mailto:marc@infomarc.info>> > > <mailto:marc@infomarc.info <mailto:marc@infomarc.info> <mailto:marc@infomarc.info <mailto:marc@infomarc.info>>>>: > > >>> > > >>> Hi Hugues, > > >>> I retested this morning on a laptop,importing a
> SQL file > > containing > > >>> 10000 employees from the sample employees > database. This is > > a small file > > >>> (660 KB). > > >>> > > >>> Current master: 3 min 25 sec (and ends with > JSON.parse: > > unexpected > > >>> character) > > >>> > > >>> Current Tithugues/stringFunctions_master: 2 min 10 > sec (same > > js error) > > >>> > > >>> Current QA_4_2: 0 min 5 sec > > >>> > > >>> There has been improvement, but we cannot release > 4.3 with > > this import > > >>> speed. > > >>> > > >>> -- > > >>> Marc Delisle | phpMyAdmin > > >> > > >> > > >> Hi, > > >> > > >> I agree… But I'm afraid this is linked to > multibytes functions… > > >> Maybe we shouldn't use the multibytesfunctions
> everywhere… > > >> > > >> I'll still try to improve performances. > > >> > > >> Hugues. > > >> > > > > > > Indeed, I also think that we should use mb_* > function only > > when necessary and choice to use them should bemade
> in case by > > case basis. > > > > > > -- > > > Thanks and Regards, > > > > > > Madhura Jayaratne > > > > Hi, > > > > I didn't push my commits, but that's what I've started. I > > replaced the mb_* calls by standard calls on configuration > > variables, reserved words, etc. > > > > Hugues. > > > > > > Hi, > > > > Out of desperation, I try another algorithm. Insteadof
> buffering > > data until SQL delimiter, I'll try to parse alllines.
> > So, I won't parse 1000 times a buffer of 50000 characters, > but less > > than 10 times many buffers of 500 characters. I hope this > will be > > faster. > > > > Hugues. > > > > > > Hi, > > > > The new algorithm is over. There are still some controlsto
> add, but it > > is usable with the file in this ticket: [1] > > You can find my modifications here: [2] > > > > Marc, is it faster for you ? > > It seems that I won ~33% of time. We're still far from 5 seconds… > > Maybe I'll try to use standard PHP functions to see the > difference. If > > the standard PHP functions are really faster, I'll try to add > an option > > to use mb_* functions or standard PHP functions, as you said. > > > > [1] https://sourceforge.net/p/phpmyadmin/bugs/4536/ > > [2] > > >https://github.com/Tithugues/phpmyadmin/tree/stringFunctions_useStandardFunc...
> > Hi Hugues, > yes it's faster. With the same testing conditions, theimport
> takes 1 > min 20 sec. > > > -- > Marc Delisle | phpMyAdmin > > > Thanks for your feedback. > I'll try another another improvement to be faster. > > Note for my self: > * read X characters but don't restart the search from 0 eachtime
> * search for the escaped quote with a lookbehind expression, > something like `(?!<\\)(\\\\)*'` > > Hugues. > > > Hi, > > As asked by Marc, I added on option to import by reading as a multibytes > string or not. > The default configuration won't read as multibytes string (because it's > too long…). It seems that the DnD to import doesn't use the default > configuration, so what ever you define as default, it won't be usein
> this process. > Should we create a ticket for this ? I think it's possible to get it in > javascript. > > Hugues. > With the new option "Read as multibytes" left to its default disabled value, my test file imports in 5 seconds. When this option is ticked, it takes 60 seconds. Note that in both cases, I get a Javascript error at the end (theerror
report does not show the same error in Firefox than in Chromium). Can anyone confirm this? -- Marc Delisle | phpMyAdminMarc,
Could we please have both errors? I don't have it on Chrome. I'll try on Firefox and IE.
Thanks, Hugues. From Iceweasel 24.8.1:
{ "pma_version": "4.3.0-dev", "browser_name": "FIREFOX", "browser_version": "24.0", "user_os": "Linux", "server_software": "Apache/2.2.22 (Debian)", "user_agent_string": "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20140924 Firefox/24.0 Iceweasel/24.8.1", "locale": "en", "configuration_storage": "enabled", "php_version": "5.4.4-14+deb7u14", "exception_type": "js", "exception": { "mode": "stack", "name": "SyntaxError", "message": "JSON.parse: unexpected character", "stack": [ { "func": ".parseJSON", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com | jquery.org/license */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": "PMA_console.initialize/<", "args": "", "line": 164, "column": "", "context": [ " $('#pma_console_options input[name=current_query]').prop('c//...", " PMA_console.updateConfig();", " });", "", " $(document).ajaxComplete(function (event, xhr) {", " var data = $.parseJSON(xhr.responseText);", " PMA_console.ajaxCallback(data);", " });", "", " PMA_console.isInitialized = true;", " }" ], "filename": "console.js" }, { "func": "ErrorReport.wrap_function/new_func", "args": "", "line": 262, "column": "", "context": [ " */", " wrap_function: function (func) {", " if (!func.wrapped) {", " var new_func = function () {", " try {", " return func.apply(this, arguments);", " } catch (x) {", " TraceKit.report(x);", " }", " };", " new_func.wrapped = true;" ], "filename": "error_report.js" }, { "func": "v.event.dispatch", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com | jquery.org/license */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": "$event.dispatch", "args": "", "line": 372, "column": "", "context": [ "$event.dispatch = function( event ){", "\tif ( $.data( this, "suppress."+ event.type ) - new Date().getTime() > 0 ){", "\t\t$.removeData( this, "suppress."+ event.type );", "\t\treturn;", "\t}", "\treturn $dispatch.apply( this, arguments );", "};", "", "// event fix hooks for touch events...", "var touchHooks = ", "$event.fixHooks.touchstart = " ], "filename": "jquery/jquery.event.drag-2.2.js" }, { "func": "v.event.add/u", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com | jquery.org/license */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": "v.event.trigger", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com | jquery.org/license */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": "v.event.trigger", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com | jquery.org/license */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": "T", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com | jquery.org/license */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" }, { "func": ".send/r", "args": "", "line": 2, "column": "", "context": [ "/*! jQuery v1.8.3 jquery.com | jquery.org/license */\r", "(function(e,t){function _(e){var t=M[e]={};return v.each(e.split(y),functio//...", "", "function sprintf() {", "/*", " * Copyright (c) 2013 Kevin van Zonneveld (http://kvz.io)", " * and Contributors (http://phpjs.org/authors)" ], "filename": "jquery/jquery-1.8.3.min.js" } ], "useragent": "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20140924 Firefox/24.0 Iceweasel/24.8.1", "incomplete": "false", "uri": "db_import.php?target=" }, "script_name": "db_import.php", "microhistory": { "pages": [ { "hash":
"index.php?db=&table=&server=1&target=&token=c455fd8a0595a5032d18c07a64e7ab0a" }, { "hash":
"db_structure.php?db=test&table=&server=1&target=&token=c455fd8a0595a5032d18c07a64e7ab0a", "params": { "opendb_url": "db_structure.php", "safari_browser": "0", "collation_connection": "utf8mb4_general_ci", "lang": "en", "server": "1", "text_dir": "ltr", "pma_text_default_tab": "Browse", "pma_text_left_default_tab": "Structure", "LimitChars": "50", "pftext": "", "confirm": "true", "LoginCookieValidity": "144000", "logged_in": "true" } }, { "hash":
"index.php?db=&table=&server=1&target=&token=c455fd8a0595a5032d18c07a64e7ab0a", "params": { "opendb_url": "db_structure.php", "safari_browser": "0", "collation_connection": "utf8mb4_general_ci", "lang": "en", "server": "1", "text_dir": "ltr", "pma_text_default_tab": "Browse", "pma_text_left_default_tab": "Structure", "LimitChars": "50", "pftext": "", "confirm": "true", "LoginCookieValidity": "144000", "logged_in": "true" } }, { "hash":
"db_structure.php?db=test&table=&server=1&target=&token=c455fd8a0595a5032d18c07a64e7ab0a", "params": { "opendb_url": "db_structure.php", "safari_browser": "0", "collation_connection": "utf8mb4_general_ci", "lang": "en", "server": "1", "text_dir": "ltr", "pma_text_default_tab": "Browse", "pma_text_left_default_tab": "Structure", "LimitChars": "50", "pftext": "", "confirm": "true", "LoginCookieValidity": "144000", "logged_in": "true" } }, { "hash":
"db_import.php?db=test&table=&server=1&target=&token=c455fd8a0595a5032d18c07a64e7ab0a", "params": { "opendb_url": "db_structure.php", "safari_browser": "0", "collation_connection": "utf8mb4_general_ci", "lang": "en", "server": "1", "text_dir": "ltr", "pma_text_default_tab": "Browse", "pma_text_left_default_tab": "Structure", "LimitChars": "50", "pftext": "", "confirm": "true", "LoginCookieValidity": "144000", "logged_in": "true" } } ], "current_index": "5" } }
-- Marc Delisle | phpMyAdmin
Hi,
It seems that the new algorithm is over, see [1]. The unit tests are now ok (except for the one failing on master). I did some tests with escaped quotes (with quote or backslash), delimiters into strings, modified delimiters. Marc, do you still have your error please? (I didn't work on it, but I hope that your issue is linked to the not-finished version…)
I don't have the error at work but I have to retest at home.
Feel free to look at the new algorithm and to tell me if you see any flaw. If you can lead to an error with a correct SQL file, please send it to me.
Thanks for your feedback.
[1] https://github.com/Tithugues/phpmyadmin/tree/stringFunctions_useStandardFunc...
Hugues.
Phpmyadmin-devel mailing list Phpmyadmin-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/phpmyadmin-devel