[Phpmyadmin-devel] #4536 - master: import problem (PMA_String)

[Hugues Peccatte]

2014-10-05 20:36 GMT+02:00 Hugues Peccatte <hugues.peccatte at gmail.com>:

> 2014-10-04 9:01 GMT+02:00 Hugues Peccatte <hugues.peccatte at gmail.com>:
>
>> Le 4 oct. 2014 03:22, "Madhura Jayaratne" <madhura.cj at gmail.com> a écrit
>> :
>>
>> >
>> >
>> >
>> > On Sat, Oct 4, 2014 at 1:24 AM, Hugues Peccatte <
>> hugues.peccatte at gmail.com> wrote:
>> >>
>> >> 2014-10-03 12:26 GMT+02:00 Marc Delisle <marc at infomarc.info>:
>> >>>
>> >>> Hi Hugues,
>> >>> I retested this morning on a laptop, importing a SQL file containing
>> >>> 10000 employees from the sample employees database. This is a small
>> file
>> >>> (660 KB).
>> >>>
>> >>> Current master: 3 min 25 sec (and ends with JSON.parse: unexpected
>> >>> character)
>> >>>
>> >>> Current Tithugues/stringFunctions_master: 2 min 10 sec (same js error)
>> >>>
>> >>> Current QA_4_2: 0 min 5 sec
>> >>>
>> >>> There has been improvement, but we cannot release 4.3 with this import
>> >>> speed.
>> >>>
>> >>> --
>> >>> Marc Delisle | phpMyAdmin
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I agree… But I'm afraid this is linked to multibytes functions…
>> >> Maybe we shouldn't use the multibytes functions everywhere…
>> >>
>> >> I'll still try to improve performances.
>> >>
>> >> Hugues.
>> >>
>> >
>> > Indeed, I also think that we should use mb_* function only when
>> necessary and choice to use them should be made in case by case basis.
>> >
>> > --
>> > Thanks and Regards,
>> >
>> > Madhura Jayaratne
>>
>> Hi,
>>
>> I didn't push my commits, but that's what I've started. I replaced the
>> mb_* calls by standard calls on configuration variables, reserved words,
>> etc.
>>
>> Hugues.
>>
>
> Hi,
>
> Out of desperation, I try another algorithm. Instead of buffering data
> until SQL delimiter, I'll try to parse all lines.
> So, I won't parse 1000 times a buffer of 50000 characters, but less than
> 10 times many buffers of 500 characters. I hope this will be faster.
>
> Hugues.
>

Hi,

The new algorithm is over. There are still some controls to add, but it is
usable with the file in this ticket: [1]
You can find my modifications here: [2]

Marc, is it faster for you ?
It seems that I won ~33% of time. We're still far from 5 seconds…
Maybe I'll try to use standard PHP functions to see the difference. If the
standard PHP functions are really faster, I'll try to add an option to use
mb_* functions or standard PHP functions, as you said.

[1] https://sourceforge.net/p/phpmyadmin/bugs/4536/
[2]
https://github.com/Tithugues/phpmyadmin/tree/stringFunctions_useStandardFunctions_master

Thanks,
Hugues.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.phpmyadmin.net/pipermail/developers/attachments/20141012/8bc5e705/attachment.html>