bbPress Standalone Importer breaks if source forum is too big

[ali3n0]

Seems there's a memory leak somewhere, given that even growing allowed memory indefinitely it still breaks pretty much at the same point. You can see the error in the attachment

[ali3n0]

Seems that here another user is facing the same problem:
http://bbpress.org/forums/topic/bbpress-plugin-20-rc-1-large-forum-import-failure

[cnorris23]

Is anyone with this problem willing/able to provide some database files to test this out? It would take some work to produce forum data of that magnitude manually. Alternatively, does anyone know of a plugin that one could use to create database info, similar to ​http://wordpress.org/extend/plugins/demo-data-creator/?

[anointed]

@cnorris23

I don't have a bbpress forum, but can provide a vbulletin forum db.
Threads: 60,982, Posts: 295,135, Members: 26,800

We also have access to forums with 10million+ posts and 500k members if that would help.

*I've currently got one of the vb devs working on building me a vb importer for bbpress plugin. He built one for standalone and it worked perfectly on the larger db, still tweaking on the plugin importer.

[johnjamesjacoby]

There is a redux of the migration tool in the works, along with migrators from several other popular forum packages.

Bumping to 2.1 so we can reevaluate this after 2.0.

[Sadr]

I can provide database files for a 20k BuddyPress legacy (1.x) forum.

[cnorris23]

I know from jjj's previous comment that this is getting some work, but just in case, I had a thought. Change the foreach statements from

foreach( $topics as $topic )

to

foreach( $topics as $key => $topic )

and at the end of the foreach run

unset( $topics[$key] )

then for good measure, run

unset( $topics )

somewhere soon after the foreach.

This should help reduce memory usage as we chug along through the import.

@jjj, if you have no objections, and it's help is still needed on this, I'd be glad to work up a patch for testing.

[johnjamesjacoby]

First pass: replace bbPress 1.1 importer with bbConverter by Adam Ellis.

• Handles large bbPress installations much better than original importer
• Better potential for new converters in future
• Breathes life into abandoned bbConverter project
• See #1592
• Props Adam Ellis for bbConverter
• @todo: Settings/UI/code clean-up, shake vigorously, lather, rinse, repeat