Ldap bulk import problems

General development questions
User avatar
cassiano.dalpizzol
Posts: 4
Joined: Mon Jan 09, 2012 7:36 pm

Ldap bulk import problems

Postby cassiano.dalpizzol » Tue Jan 10, 2012 1:55 pm

Hello,

I'm one of those developers cited on these post.

We do have some problems importin more than 11.000 users in a single batch, but using the cli interface of the setup (as seein in this post) resolved some (but not all) of our problems.

The problem is some untreated exceptions on the usersync backend. Since our Ldap and the application BD are not equal sometimes the import user script dies on the insert data with messages like

"String data, right truncated: 1406 Data too long for column 'tel_fax' at row 1"

and we have to start all over again and, belive me, 11000 users are not imported as fast as we hoped :lol:

I have managed to fix this (for us) changind the code in the Tinebase_User::syncUsers method by adding a second catch block to the original one and loging the problem and moving on to the next user.

(public static function syncUsers)

Code: Select all

foreach($users as $user)
{
    try
    {
        $user = self::syncUser($user, $_syncContactData);
    } catch (Tinebase_Exception_NotFound $ten) {
        Tinebase_Core::getLogger()->crit(__METHOD__ . '::' . __LINE__ . " User {$user->accountLoginName} not synced: "
            . $ten->getMessage());
    } catch (Exception $e) {
         Tinebase_Core::getLogger()->crit(__METHOD__ . '::' . __LINE__ . " User {$user->accountLoginName} not synced: "
            . $ten->getMessage());
    }
}


Since it will crash the application if this new layer of catch is not applied on our context it resolved the problem. Did you guys see another problem arising because of this implementation?

And since the same piece of code is used in the catch block, why not to just change the type of exception in the first one and get rid of the later (treat only the outermost exception)?

Thanks in advance
Cassiano Dal Pizzol

Return to “General”

Who is online

Users browsing this forum: No registered users and 1 guest