Switching databases causes crashes

[StephenUK]

I have been working with one database, but now that it has reached 700 Megabytes it seemed prudent to move some data into a second database to keep size down. This has been achieved without problem.

However, I am finding that the program crashes when I switch between the databases, whether by the f6 command, the window menu, or the database toolbar (which only sometimes displays the database names).

Am I better off sticking with one database and letting it continue growing instead? I doubt it would ever get bigger than about 3 Gigabytes.

[janrif]

Stephen I will be interested in Kinook's answer as I, too, will eventually be dividing up my DB.

Unlike Zoot, the problem w this is that URp doesn't search across DBs (which I assume you know) so you sort of have to remember what is where. I'm not good @ that.

Of course it's easier to backup, etc when you are workng with a single DB.

When I broached this w Kinook as I first started using URp it was suggested that I consider a single DB. I did that & in fact, that DB is currently named all_my_stuff. :-)

IMO & in any case, user should be able to break DBs up as they wish & be able to switch between them w/o issue & there should be a global search function to make it convenient.

I figure if Zoot can do it in a simpler architecture, URp should be capable of this as well.

[StephenUK]

Jan - I agree the problem of remembering which database one put something in, which, as you say, is more of a problem if one can only search one database at a time.

Given the excellent hoist / favorites command which tames a large data tree, my preference is to have one large database. However, I am nervous going too big. Cetainly I was having no problems with about 700 Megabytes.

As an aside, I use VersionBackup to produce a fresh copy of the database each day going back ten days, so perhaps I don't have too much to lose by letting the database go on getting bigger and bigger. If it ever really falls over irretrievably, perhaps that is the time to divide it up...

Nonetheless, it would be good to have some guidance. And to know why it crashes when I switch...

[kinook]

Databases that large or larger shouldn't be a problem, but using multiple databases should also work. We regularly run with multiple databases open at once and are able to switch between them without errors or crashes. Is the problem specific to your databases (can you, for instance, open a few sample databases and switch between them?)? Please ZIP and send or post:
1) The info from Help | About | Install Info
2) Run RegEdit and export the registry key "HKEY_CURRENT_USER\Software\Kinook Software\Ultra Recall"
3) The files at "%APPDATA%\Kinook Software\Ultra Recall"
4) A screen shot or exact error message when the crash occurs

Thanks.

[zargron]

700 MB – that’s quite a decent size StephenUK! My UR implementation strategy avoids letting databases grow to more than about 10% of the size of yours. I presume you have a reasonable amount of stored content? Hey – if you don’t mind – what are your database stats? eg. from menu [File | Properties].

To keep database size down I link rather than store files (as a few of us discussed in thread http://www.kinook.com/Forum/showthread.php?s=&threadid=1031). I also admit to being a compulsive compartmentalizer. To that end I’m likely to create separate “resource” orientated databases for big projects. (BTW: I have one central DB covering PIM type usage.) With separate databases, I’m in a better position to archive a specific project with all it’s key data by simply archiving the project's main folder.

As alluded to by Kinook, UR seems to handle multiple databases fine - I use it a lot - love that <F6> key! However, not with big DB's. I would also support a search multiple database function. So, if anyone has some constructive comments on how to implement such a facility – perhaps start a thread in the Suggestions area?

[StephenUK]

Kinook - many thanks for the feedback. I will let you have that information.

Zargron. Thanks for the reference to the thread. My database size indeed comes mainly from storing things. The bulk of it is web site captures where I want more than a link because of changing content. I am not sure if I can export those captures so as then to link with them, and I would anyway lose the ability to search within UR on the linked items.

I have been a very keen user of InfoSelect since its DOS days. The reason for the (partial) switch to UR is so that I can put everything in one place. I can link quite effectively from InfoSelect, so really it is the ability to avoid mere linking that attracts me to UR.

I did use WebResearch for web capture and found it very good, but then I found no easy way to associate web captures with photos, notes and Word files held elsewhere. Essentially I need an integrated approach to diverse types of information. So UR is, I think, right for me and is in my view a very intelligently conceived program. I like its features, especially the cloning and hoisting which are not really available in InfoSelect. The only issue for me with UR is one of stability.

Maybe future development should concentrate as much on stability and safety as much as on further features.

One thought is that if there were on-the-fly duplication of everything in a directory structure in a backup directory, then I would feel reassured that I always had that to fall back on. (And to bang an old drum of mine, if Paperport were then used to look at that data it is a much nicer interface which I wish Kinook would emulate. Paperport creates thumbnails of everything including Word and Excel files. Searching visually is a very attractive alternative for many documents which often are not given very meaningful names. Mind you, Paperport makes a real mess of creating thumbnails of web pages...).

Perhaps the surprising thing is that applications such as UR, which to my mind ought to be central to PC use, still have only small acceptance.

[zargron]

Thanks for the enlightening comments Stephen.
Maybe future development should concentrate as much on stability and safety as much as on further features.Back in May of 2000 during the initial release of SQLite I bet no-one pondered that 7 years later you couldn't buy a computer with less than a 100GB hard disk and that some people would be using SQLite for 1GB+ database files. I'll take a punt here by suggesting that with UR being so good it's easy for users to build up 500MB+ files which then push the limits of SQLite. In a normal relational database scenario SQLite would be fine. Drop it into a hyperactive application like UR and it gets a little testy.

If you want 99% stability now - keep your file sizes down. If you want 99.9% stability and large .urd files in the future - a couple of things need to happen. First a change in database engine. Perhaps over the next 12 to 24 months UR could be ported to run against a stronger DB engine. However, we will very likely pay a performance penalty! The other change is unrealistic - provide a UR version to run under a Unix / Linux based operating system. :D
Perhaps the surprising thing is that applications such as UR, which to my mind ought to be central to PC use, still have only small acceptance.Yes. The crux here is the ever present challenge of information management. It's hard enough for the average person to manage information when the structure is laid out for them. If you give them an open palette like UR then you best make sure the oxygen bottle is handy...

[StephenUK]

Zagron - very interesting comments on the database engine. I'm not, however, 100% convinced that a better database would improve things. Looking at the posts elsewhere no-one has actually demonstrated that SQLite can't cope with large databases! There is just a nagging doubt creaping into posts.

Maybe it would be easy for Kinook to do a little testing with databases of up to, say, 20 Gig. Then they could, for example, produce a grid showing the tradeof in terms of lost performance, liability to crash, etc. That way we would know where we stand. At the moment the slightly confusing message seems to be "the database can be as big as you like, but it's best to keep it small if you can".

[zargron]

On the topic of testing...
Asking your average software development house to spend a whole heap of hours limit testing their software is sound in theory but not always practical. The good ones do a reasonable amount of in-house testing before releasing it to a beta community for some "real-life" testing. Hopefully some of the beta testers push it to reasonable limits. And still it's only after the general release that you really find out where you're at. People accidentally or deliberately do strange things with your software and POP! Your cosy in-house testing didn't pick it up because it's quite hard to get people who know the software well to do the wrong thing with it.

I reckon Kinook are taking the stability thing very seriously. They are continuously absorbing the feedback and chipping away at it. Unless you've got millions of dollars to throw at it - complex software takes a long time to get clean. And BTW - there is the issue of the operating system. :D

Now what else was I going to mention? Oh - that's right - SQLite and limitations. I agree. The SQLite engine might be quite blameless. Perhaps it is in Kinook's implementation of it. There's just so many factors - again - it simply takes time to track them down and deal with them. - (pause) - Just back from the SQLite site. For what its worth, I took the following from:
http://www.sqlite.org/limits.html
SQLite will support very large databases in theory, but the current implementation is optimized for the common SQLite use cases of embedded devices and persistent stores for desktop applications. In other words, SQLite is designed for use with databases sized in kilobytes or megabytes not gigabytes. If you are building an application to work with databases that are hundreds of gigabytes or more in size, then you should perhaps consider using a different database engine that is explicitly designed for such large data sets.

[StephenUK]

Zargron - good points about software developers. They have an almost impossible task and one has to take one's hat off to Kinook as a small company for producing something better in its own way, than, say, OneNote, with a tiny fraction of the resources.

In fact the SQLite quotation is kind of reassuring. Sounds like a few Gig ought to be OK and that's enough for me. Nonetheless, if one were looking for a network version I think something more powerful would be needed. I guess there are tradeoffs with everything.

I think your point about software developers finding it difficult to make mistakes in their own software applies also to the UR help file. Things that are obvious to them get missed out, like how to create a note or the purpose of the document template... But finding out for oneself is perhaps half the fun. At least with UR there is nearly always an answer or workaround somewhere.

[kinook]

I don't think this problem has anything to do with the size of the database(s). It looks similar to http://www.kinook.com/Forum/showthread.php?threadid=2344, so changing 'Tools | Options | General | Windows taskbar entries' to 'Always one for each open database' might work better for now.

I was not able to reproduce the problem with the files you sent, so we'll probably need to create a debugging build for you to run to help isolate the problem.

[StephenUK]

Kinook - many thanks! Yes, that solved it!

[zargron]

Excellent Kinook - well done again!

I suggest you post an entry into "FAQ - Troubleshooting" ASAP that conveys this fix and gives more information on the impact of "Windows taskbar entries" in [Tools | Options... | General].

[StephenUK]

Zargron, yes, good idea. I will.

[StephenUK]

Zargron - just tried to post under FAQ, but I think I don't have the right to post in that section.

[zargron]

Originally posted by StephenUK
Zargron - just tried to post under FAQ, but I think I don't have the right to post in that section. Sorry Stephen, my suggestion to post in "FAQ - Troubleshooting" was directed at Kinook.

[cnewtonne]

my 2 pennies ...

- it is way better for any software to have its bugs caught and fixed while in beta testing. Users expect it and will appreciate the fact that they were given the chance to test it. If, however, same set of bugs are caught after it is declared 'production', the attitude, image, and perception will be quite different. Users will get disappointed, demote the product, and look elsewhere. Notice that we are talking same set of bugs, yet very different outcomes.

Having said that, I do strongly recommend that kinook do come up with a beta testing group external to their own labs. Why not release a beta version with all traces enabled. Ask users to beat the crab out of it and report their findings. Not only that, you can even take it a step farther divide up the work....
- have a set of users test the OL integration piece.
- Test the item management piece e.g. moving, copying, etc...
- and so forth.

all this work should be done in less than month max. I myself participate in 3 beta groups for different kinds of products and I find quite productive and helpful.

[StephenUK]

Maybe a beta testing group would be a good idea.

However, it does seem to me that perhaps we sometimes expect too much. The software only costs the price of a meal out, and it is relatively niche. The developer clearly works extremely hard and is very responsive. Maybe with this kind of complex product we shouldn't come down too hard on the odd thing going wrong now and again. Which is not to say I don't agree with what you say!

[StephenUK]

Version 3.2 has completely fixed the switching problem, in all modes. Thanks Kinook.

[CurtC]

I'm having this issue, UR goes not responsive when I switch between databases. Tried setting it to show a taskbar entry for each open DB, same result, both went unresponsive.

Ultra Recall Professional 3.2.6
Registered to: Curt Coulter (1-user license)
Windows version: 5.1.2600.2.0
Install path: C:\Program Files\UltraRecall
EncryptPDF.dll version 3.0.0.2
mimepp.dll version 3.0.4
msptls.dll version
pdf2txt.dll version 3.1.0.4
PolarSpellChecker.dll version 4.0.5.4
riched20.dll version 12.0.4518.1014
SftPrintPreview_IX86_U_10.dll version 1.05
SftTree_IX86_U_50.dll version 5.06
UltraRecall.exe version 3.2.6.3
unins000.exe version 51.49.0.0
Database filename: C:\Documents and Settings\curt\My Documents\CBORD Dev.urd
Database version: 3.0.13

[kinook]

Weird. Is this something that used to work and stopped working? Try closing UR and deleting C:\Program Files\UltraRecall\riched20.dll. Does the problem occur with the sample databases? Do you have much resident software running (i.e., security software, desktop enhancers, system tray apps, etc.) and if so, what products+versions?

[CurtC]

Originally posted by kinook
Weird. Is this something that used to work and stopped working? Try closing UR and deleting C:\Program Files\UltraRecall\riched20.dll. Does the problem occur with the sample databases? Do you have much resident software running (i.e., security software, desktop enhancers, system tray apps, etc.) and if so, what products+versions?

Yes it used to work. I've rebooted, closed out all other apps, and it has the same behavior. Removing riched20.dll, same result.

Opened 3 sample databases, I can switch between them at will, no problem. I did a compact & repair on all 3 of my DBs, same result.

After trying the sample DBs, but having mine still fail, I closed everything, opened them each individually and closed them. Then opened all three again, and now I can switch between them with no problem. Strange.

Thanks for the help.