Ldap bulk import problems

General development questions

Ldap bulk import problems

Postby cassiano.dalpizzol » Tue Jan 10, 2012 2:55 pm

Hello,

I'm one of those developers cited on these post.

We do have some problems importin more than 11.000 users in a single batch, but using the cli interface of the setup (as seein in this post) resolved some (but not all) of our problems.

The problem is some untreated exceptions on the usersync backend. Since our Ldap and the application BD are not equal sometimes the import user script dies on the insert data with messages like

"String data, right truncated: 1406 Data too long for column 'tel_fax' at row 1"

and we have to start all over again and, belive me, 11000 users are not imported as fast as we hoped :lol:

I have managed to fix this (for us) changind the code in the Tinebase_User::syncUsers method by adding a second catch block to the original one and loging the problem and moving on to the next user.

(public static function syncUsers)
Code: Select all
foreach($users as $user)
{
    try
    {
        $user = self::syncUser($user, $_syncContactData);
    } catch (Tinebase_Exception_NotFound $ten) {
        Tinebase_Core::getLogger()->crit(__METHOD__ . '::' . __LINE__ . " User {$user->accountLoginName} not synced: "
            . $ten->getMessage());
    } catch (Exception $e) {
         Tinebase_Core::getLogger()->crit(__METHOD__ . '::' . __LINE__ . " User {$user->accountLoginName} not synced: "
            . $ten->getMessage());
    }
}


Since it will crash the application if this new layer of catch is not applied on our context it resolved the problem. Did you guys see another problem arising because of this implementation?

And since the same piece of code is used in the catch block, why not to just change the type of exception in the first one and get rid of the later (treat only the outermost exception)?

Thanks in advance
Cassiano Dal Pizzol
User avatar
cassiano.dalpizzol
 
Posts: 4
Joined: Mon Jan 09, 2012 8:36 pm

Return to General

Who is online

Users browsing this forum: Google [Bot] and 1 guest

Startseite
NewsDemoDownloadForumWikiBlog
Support
Support at first hand!
If the forum does not help anymore ... Professional support is available directly from our Tine2.0 core Developers.

more »