<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">2014-10-19 13:21 GMT+02:00 Marc Delisle <span dir="ltr"><<a href="mailto:marc@infomarc.info" target="_blank">marc@infomarc.info</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Le 2014-10-18 14:48, Hugues Peccatte a écrit :<br>
<span class="">> 2014-10-13 12:21 GMT+02:00 Hugues Peccatte <<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a><br>
</span>> <mailto:<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a>>>:<br>
<span class="">><br>
> 2014-10-12 21:06 GMT+02:00 Marc Delisle <<a href="mailto:marc@infomarc.info">marc@infomarc.info</a><br>
</span>> <mailto:<a href="mailto:marc@infomarc.info">marc@infomarc.info</a>>>:<br>
<span class="">><br>
> Le 2014-10-12 12:57, Hugues Peccatte a écrit :<br>
> > 2014-10-05 20:36 GMT+02:00 Hugues Peccatte <<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a> <mailto:<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a>><br>
</span>> > <mailto:<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a><br>
<span class="">> <mailto:<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a>>>>:<br>
> ><br>
> > 2014-10-04 9:01 GMT+02:00 Hugues Peccatte <<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a> <mailto:<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a>><br>
</span>> > <mailto:<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a><br>
<span class="">> <mailto:<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a>>>>:<br>
> ><br>
> > Le 4 oct. 2014 03:22, "Madhura Jayaratne" <<a href="mailto:madhura.cj@gmail.com">madhura.cj@gmail.com</a> <mailto:<a href="mailto:madhura.cj@gmail.com">madhura.cj@gmail.com</a>><br>
</span>> > <mailto:<a href="mailto:madhura.cj@gmail.com">madhura.cj@gmail.com</a><br>
<span class="">> <mailto:<a href="mailto:madhura.cj@gmail.com">madhura.cj@gmail.com</a>>>> a écrit :<br>
> ><br>
> ><br>
> > ><br>
> > ><br>
> > ><br>
> > > On Sat, Oct 4, 2014 at 1:24 AM, Hugues Peccatte<br>
> > <<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a><br>
> <mailto:<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a>><br>
</span>> <mailto:<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a><br>
<span class="">> <mailto:<a href="mailto:hugues.peccatte@gmail.com">hugues.peccatte@gmail.com</a>>>><br>
> > wrote:<br>
> > >><br>
> > >> 2014-10-03 12:26 GMT+02:00 Marc Delisle <<a href="mailto:marc@infomarc.info">marc@infomarc.info</a> <mailto:<a href="mailto:marc@infomarc.info">marc@infomarc.info</a>><br>
</span>> > <mailto:<a href="mailto:marc@infomarc.info">marc@infomarc.info</a> <mailto:<a href="mailto:marc@infomarc.info">marc@infomarc.info</a>>>>:<br>
<div><div class="h5">> > >>><br>
> > >>> Hi Hugues,<br>
> > >>> I retested this morning on a laptop, importing a<br>
> SQL file<br>
> > containing<br>
> > >>> 10000 employees from the sample employees<br>
> database. This is<br>
> > a small file<br>
> > >>> (660 KB).<br>
> > >>><br>
> > >>> Current master: 3 min 25 sec (and ends with<br>
> JSON.parse:<br>
> > unexpected<br>
> > >>> character)<br>
> > >>><br>
> > >>> Current Tithugues/stringFunctions_master: 2 min 10<br>
> sec (same<br>
> > js error)<br>
> > >>><br>
> > >>> Current QA_4_2: 0 min 5 sec<br>
> > >>><br>
> > >>> There has been improvement, but we cannot release<br>
> 4.3 with<br>
> > this import<br>
> > >>> speed.<br>
> > >>><br>
> > >>> --<br>
> > >>> Marc Delisle | phpMyAdmin<br>
> > >><br>
> > >><br>
> > >> Hi,<br>
> > >><br>
> > >> I agree… But I'm afraid this is linked to<br>
> multibytes functions…<br>
> > >> Maybe we shouldn't use the multibytes functions<br>
> everywhere…<br>
> > >><br>
> > >> I'll still try to improve performances.<br>
> > >><br>
> > >> Hugues.<br>
> > >><br>
> > ><br>
> > > Indeed, I also think that we should use mb_*<br>
> function only<br>
> > when necessary and choice to use them should be made<br>
> in case by<br>
> > case basis.<br>
> > ><br>
> > > --<br>
> > > Thanks and Regards,<br>
> > ><br>
> > > Madhura Jayaratne<br>
> ><br>
> > Hi,<br>
> ><br>
> > I didn't push my commits, but that's what I've started. I<br>
> > replaced the mb_* calls by standard calls on configuration<br>
> > variables, reserved words, etc.<br>
> ><br>
> > Hugues.<br>
> ><br>
> ><br>
> > Hi,<br>
> ><br>
> > Out of desperation, I try another algorithm. Instead of<br>
> buffering<br>
> > data until SQL delimiter, I'll try to parse all lines.<br>
> > So, I won't parse 1000 times a buffer of 50000 characters,<br>
> but less<br>
> > than 10 times many buffers of 500 characters. I hope this<br>
> will be<br>
> > faster.<br>
> ><br>
> > Hugues.<br>
> ><br>
> ><br>
> > Hi,<br>
> ><br>
> > The new algorithm is over. There are still some controls to<br>
> add, but it<br>
> > is usable with the file in this ticket: [1]<br>
> > You can find my modifications here: [2]<br>
> ><br>
> > Marc, is it faster for you ?<br>
> > It seems that I won ~33% of time. We're still far from 5 seconds…<br>
> > Maybe I'll try to use standard PHP functions to see the<br>
> difference. If<br>
> > the standard PHP functions are really faster, I'll try to add<br>
> an option<br>
> > to use mb_* functions or standard PHP functions, as you said.<br>
> ><br>
> > [1] <a href="https://sourceforge.net/p/phpmyadmin/bugs/4536/" target="_blank">https://sourceforge.net/p/phpmyadmin/bugs/4536/</a><br>
> > [2]<br>
> ><br>
> <a href="https://github.com/Tithugues/phpmyadmin/tree/stringFunctions_useStandardFunctions_master" target="_blank">https://github.com/Tithugues/phpmyadmin/tree/stringFunctions_useStandardFunctions_master</a><br>
><br>
> Hi Hugues,<br>
> yes it's faster. With the same testing conditions, the import<br>
> takes 1<br>
> min 20 sec.<br>
><br>
><br>
> --<br>
> Marc Delisle | phpMyAdmin<br>
><br>
><br>
> Thanks for your feedback.<br>
> I'll try another another improvement to be faster.<br>
><br>
> Note for my self:<br>
> * read X characters but don't restart the search from 0 each time<br>
> * search for the escaped quote with a lookbehind expression,<br>
> something like `(?!<\\)(\\\\)*'`<br>
><br>
> Hugues.<br>
><br>
><br>
> Hi,<br>
><br>
> As asked by Marc, I added on option to import by reading as a multibytes<br>
> string or not.<br>
> The default configuration won't read as multibytes string (because it's<br>
> too long…). It seems that the DnD to import doesn't use the default<br>
> configuration, so what ever you define as default, it won't be use in<br>
> this process.<br>
> Should we create a ticket for this ? I think it's possible to get it in<br>
> javascript.<br>
><br>
> Hugues.<br>
><br>
<br>
</div></div>With the new option "Read as multibytes" left to its default disabled<br>
value, my test file imports in 5 seconds.<br>
<br>
When this option is ticked, it takes 60 seconds.<br>
<br>
Note that in both cases, I get a Javascript error at the end (the error<br>
report does not show the same error in Firefox than in Chromium). Can<br>
anyone confirm this?<br>
<span class=""><br>
<br>
--<br>
Marc Delisle | phpMyAdmin<br></span></blockquote><div><br></div><div>Marc,</div><div><br></div><div>Could we please have both errors? I don't have it on Chrome. I'll try on Firefox and IE.</div><div><br></div><div>Thanks,</div><div>Hugues. </div></div></div></div>