Import failure

[Jon Polish]

I know this problem has been mentioned before, but I would like some clarification as to its cause, limitations and work-around.

Importing and exporting has been aggratating problems for me. I do both routinely and both procedures involve large amounts of information. I have been told that URp is simply not designed to handle large imports and exports, but I do not know what the limits are. Besides, with other programs able to do both effortlessly, I fail to see why this should not be addressed in this otherwise very impressive product.

Knowing that imports consume 100% CPU usage and take FOREVER to boot, I choose (inconveniently) to start imports on Friday evenings. URp displays a dialogue indicating that it is working, but the next day, nothing is imported and there is no error message.

The current failure involves a note delimited rtf that is approximately 24 mb. Other files have failed that were significantly smaller, but contained more delimiters. For example, one failure was with a 6 mb txt file containing over 20,000 delimited notes.

It is far too time consuming to get information in and out of UR. This just adds insult to injury.

Does anyone have any advice? It would be greatly appreciated

Jon

[kinook]

If there any way you could provide an RTF file and details on how you imported that result in this problem (ZIP and send to support@kinook.com)? Thanks.

[Jon Polish]

The contents of the rtf file are confidential, so I cannot provide them. Sorry.

However, I can provide details on the import attempt.

1. Created a folder under the root (My Data) and named it 2007-05
2. File | Import (A Text of Rich Text (RTF) File
3. Browse to the file to be imported
4. Click String
5. Check Match case and Remove String
6. Choose "Selected Item in Data Explorer Pane"
7. Uncheck "Create logically linked..." and check "Automatically skip unreachable URLs."
8. Click "Finished."

A dialogue saying "Please Wait" pops up. After a very, very, very long time it goes away. Nothing was imported.

Jon

[kinook]

I crufted up a 25MB RTF file with about 25,000 delimited items and imported in the manner specified. It did take a while to import, but it always completed successfully (all items imported) and only consumed 50% CPU (on a dual-core AMD system) while importing.

[Jon Polish]

Thank you for testing.

I'm stumped. Mine definitely does not work, although it initially appears to. That is, I see the dialogue, which remains on screen while UR chugs along. After that, nothing. No error message appears and the imported items do not appear in the database. This is repeatable.

As for CPU usage, I have tried it on a dual core system. I confirm the 50% usage. But if my current single core systems can run Vista Premium, why should I have to upgrade my hardware to use UR?

Jon

[kinook]

It is odd. Is there an 'Import document(s) under <item name>' item at the top of the Undo toolbar drop-down afterwards? Does searching on items with a Date Created of today show anything?

You could try lowering the process priority (run taskmgr, right-click on ultrarecall.exe -> Set Priority -> Below normal [or Low]) before importing. But if you're importing that much data regularly, you probably want the fastest dual- or quad-core system you can find.

[Jon Polish]

Originally posted by kinook
Is there an 'Import document(s) under <item name>' item at the top of the Undo toolbar drop-down afterwards? Does searching on items with a Date Created of today show anything?

No to the first and second questions.

Jon

[Jon Polish]

Kinook:

I have put together an example of a file I am trying unsuccessfully to import. Much of the file has been converted to gibberish, but it will demonstrate the problem I am experiencing. The steps I take to import this file (and others like it) are listed above. I will email this to you at support@kinook.com.

This file is a text only export of approximately 29,000 emails. The delimiter is ==END==

Thank you.

Jon

[kinook]

Thanks, I was able to reproduce the silent failure with your test file, and the main download has been updated with a patch for that problem. But it will be faster to import that large of file via CSV import rather than text/RTF. I've attached a script which will convert email.txt to CSV format as email.txt.csv, which can then be imported via the CSV option (uncheck 'First row contains field names' and map Field 1 -> Item Text).

[Jon Polish]

Holy (never mind, I don't want to get banned)!

Importing as a csv works much better. Not blazingly fast, but I can live with this. Thank you so much.

With regard to the fix for the silent failure, can you tell me precisely what this fixes? Do you mean that I can now successfully complete an import of similar files or do you mean that UR will notify me of its failure to import these files?

Jon

[kinook]

Originally posted by Jon Polish
With regard to the fix for the silent failure, can you tell me precisely what this fixes? Do you mean that I can now successfully complete an import of similar files or do you mean that UR will notify me of its failure to import these files?

It will no longer silently fail (do nothing) when loading a large text file like that, and it should successfully import the file (eventually).

[Jon Polish]

Originally posted by kinook
It will no longer silently fail (do nothing) when loading a large text file like that, and it should successfully import the file (eventually).

OK, that is good.

My only wish is that the overall performance be improved. Thank you for fixing this very annoying problem for me.

Jon