Kinook Software Forum

Go Back   Kinook Software Forum > Ultra Recall > [UR] General Discussion
Register FAQ Community Calendar Today's Posts Search

Reply
 
Thread Tools Rate Thread Display Modes
  #1  
Old 04-08-2024, 01:24 PM
Spliff Spliff is offline
Registered User
 
Join Date: 04-07-2021
Posts: 207
( QS/AS being QuickSearch / Advanced Search, respectively, and obviously: )

A)

In practice, it's NOT possible to "flatten" the hierarchy in UR, since:

a) compare this construct:

'a...
____.b...
________Something:

.b has 'a (or more precisely: \'a) in its lineage, and
Something has both \'a... and \.b... in its lineage, and
Something, and also anything within the possible sub-tree of Something, can be found by QS (triggered from 'a... or from .b) Something (with "LS" checked), "LS" being the checkbox "Limit search to selected item in Data Explorer pane", in other words, "Local search", i.e. "just in the current sub-tree";

b) with this construct:

.a - ...
.ab - ...
____Something:

even .ab (which corresponds to above's 'a .b, obviously) does NOT have .a (or then, \.a) in its lineage, all the less so would Something have \.a in its lineage, thus:
triggering, from .a, an "LS" QS search, for .ab - ... or any of .ab -...'s child items (including Something, and any of Something's child items, and so on), would fail; you would have to do a global search instead, triggering lots of unwanted "finds", in case.

Obvious solution to this problem: You build an AS instead (if the * works on your system that is), with:
- item - contains keywords - Something (or whatever string the "target" item may contain, in its title, content, "Notes", etc., anywhere below .ab in general, or below Something in particular)
- AND - Lineage - matches wildcard - \a* (or \ab* if you want to do an even more specific search)

Thus, with strict technical (=indentation-level) hierarchy only, you can avoid AS, whilst with just a "virtual" hierarchy", .a and .a[a-z] being placed onto the same indentation level, you will need the AS... which is a nightmare if you do perhaps hundreds of "searches" (i.e. "greps", filterings) a day (which is my case).

B)

But this "flattening out" the hierarchy in UR would be of interest indeed, considering that:

Whenever you want to [/B]navigate[/B] to some "item" (i.e. some "inter-title", prefixed with some "marker", as the ' and . used above), it's evident that a simple entry .a, or then .ar will be oh so much more "easy" - if those "."-entries are immediately below your source item, i.e. in level 1, than to navigate first to some entry 'a, then to its child item .b (without knowing if item 'a has already been expanded or not)... and then, if you are mistaken of the existence / naming of an item .b "within" parent range 'a, and if some other range 'whatever is expanded at the same time, your macro will "target" the .b... inter-title within, let's say, the 'c... parent range, since your macro will use the in-built tree-navigation, by "targeting" the "very next" .b... item, be that within its intended range 'a..., or then within the 'c... range if that one is expanded, and some .b... item is existant within that range...

And yes, you can then, i.e. "after the facts", trigger a "UR command line" retrieval, and then check if your ".b..." find (or further find below .b...) has a \'a within its lineage, but that costs 1,000 or 1,500 ms more (since your macro will have to retrieve, by clipboard, that "UR command line" first, before being able to analyse it), and this procedere cannot be called "elegant" or "smooth" in any way...

C)

On the other hand, within the file system, such macros are without problems, since in both situations, a) and b), you just work with * (i.e. a*\b*\*/something or then, ab*\*/something) and thus you have free choice between the paradigms "deep" or "flat", and both procedures work fine, by any professional code: I do both, i.e. all four, both in AHK and in Everything, the * being a placeholder for some sub-routine in case but which, on a modern PC, just takes some ms indeed, and NO markers like ' or . being needed, since the \ will lead the code, whilst in a db like UR, the LEVEL is ambiguous, i.c. can not be counted fast by the number of \ within the path, and then, the (navigational, move, copy, link) TARGET is specified by that path, and reached immediately, whilst in a db like UR, by scripting from the outside, it will have to be reached, step-by-step, and time-consuming path verifications being necessary at every such step then.

D)

Thus, yes, in DBs like UR, and especially in such DBs, a "flattened" hierarchy is largely preferable (since it does away with the need for "command line" verification, "is the target .b... really a child item of intermediate target 'a...?", all these not being needed within a file system navigation / move / etc), but, frankly spoken, UR's AS "costs" you several seconds "on top" of what would have been necessary, for every single search, and thus, if there was available some ENHANCED search box, as in MI and other software, and which would allow for entering
l:a* Something [l: = "within lineage", or better: p: for "path"]
or then, if you want it to be more precise:
l:ab* Something

this would not only speed up searching / filtering enormously, but it would also spare you to first navigate to the "parent item" in question, then check the "LS" (see above), but you could trigger this QS (as you could trigger the respective, but enormously time-consuming AS now) from anywhere, and a search "from current item", i.e. within the sub-hierarchy of that current item, could be triggered by:
l: Something
i.e. without placing any value behind the l:

And similar for any other QS:

Kyle, could you please consider such an enhanced Search Box, but with publishing a list of all its codes you consider to implement / use in it, in order for allowing us to have a look upon them, and to give our view?

And yes, such a system would sacrify the possibility for searching for some:some, but then, nobody really would need to search for some:some, except if they had used that string-form for tags, and why couldn't they replace those tags then by some.some indeed?

And yes, the some before the : should be as short as possible (hence my suggestion to publish them here, before implementation, or then, why not invite us to make such list, open to suggestions then, from the current "Item/Attribute" drop-down):

pt:some (always without any space around the ":") would be parent-title:some, and
pt: would be parent-title:current-item's-title, as any empty value after the : would be "current-value" wherever it makes sense.

The necessary "core" code for such a UR Search Revolution had already been implemented, by your transcription from AS Search grid to SQL string, and, frankly, UR's current AS Search grid is a nightmare, costing an additional 5s even when applying a script, and costing minutes for manual AS search: the absence of a FAST entry for the variable part(s) of those "stored searches" being the main problem, the "grouping by indentation instead of by parentheses" being the secondary one...

and the good news is, twofold:
- user's parentheses within the enhanced search box could be transferred 1-by-1 to the sql code (i.e. no additional coding work needed), and
- the current AS grid could be preserved without any additional effort, for people who prefer that, well, "very original" (and very time-consuming), way of searching. ;-)
Reply With Quote
  #2  
Old 04-08-2024, 10:30 PM
kinook kinook is online now
Administrator
 
Join Date: 03-06-2001
Location: Colorado
Posts: 6,027
Regarding the original topic about AND and OR searches not working, I did find a bug. There is logic to support case-insensitive Unicode searching (see SearchNonAsciiTextNonCaseSensitive), which is enabled by default, and this logic lowercases the indexed text and search terms that are entered, and it was incorrectly lowercasing AND and OR in the search phrase, which prevented the FTS search engine from treating these as AND and OR search clauses. This has been fixed in the latest download (v6.3.0.14). A workaround is to disable that registry setting if you don't need case insensitive Unicode search capability.

Regarding the other topics discussed here, can you boil this down to plain English? I am not quite following what you wrote.

Note: Updated in v6.3.0.15 to also handle NOT and NEAR commands.
Reply With Quote
  #3  
Old 04-09-2024, 06:19 AM
Spliff Spliff is offline
Registered User
 
Join Date: 04-07-2021
Posts: 207
A)

I hope my previous post is understandable, though, when detailing the problems which arise with navigation in level hierarchy (equivalent to the file system's folder hierarchy, and in which the user doesn't encounter those problems, so that in order to do RELIABLE navigation (which is even more important for moving / copying items), a flattened (or you might call it virtual) hierarchy is preferable.

Btw and inadvertently, I had left out the main argument for that flattened hierarchy (construct b instead of construct a):

Construct a)

'a (all these expanded currently, or even just to be expanded by navi-macro)
__.a
____be it some item here
__.b (the .marker assures you get here, instead of the "b" item before)*
__.d (they can be grouped by subject, no abc-sorting needed)
__etc
''b
__.a
__.c

Now, the user wants to "go" to 'a .d:
> (s)he will enter ad, the macro does rest, and (s)he gets to the .b "in" 'a
> perfect

(And you wanted to go to the "b..." item, you entered aab instead, the macro navigating to 'a..., then to .a..., then to b...)

But the user is mistaken, wants to go to 'a... .d... in fact, but thinks that .d... starts with c ((s)he will have named the .d... by some synonym .c... but fails to remember):
> enters ac, expects to get to 'a .c (but that doesn't exist)
> boom: will arrive at 'b .c instead: Not too much harm for simple navigation, but really awful for moves! (Given the fact that all such navigation is done by character entry within the Data Explorer.)

Now the same situations in construct b):
.a
.aa
__be it some item here
.ab
.ad
.aetc
.b
.ba
.bc

You enter a, the macro goes to .a; you enter ab, the macro goes to .ab; you enter aab, the macro goes to .aa, then to the b-item (child of .aa).

But now, if the user is mistaken, thinks .ad is called .ac, (s)he enters ac or acsomething in order to get even deeper into .ac - and (s)he will NOT "land" somewhere in .b but in .a (since .ac doesn't exist): You "land" (or move something as last item into) the parent range, where you (or the moved item(s) will be much more "at home" than you, or they, will be somewhere within .b.

In the file system, that's no problem, since any path element's (=intermediate folder's) existence will be checked, and there is does not exist, the macro stops, and goes to (or puts into) the nearest parent folder (which does exist): These verifications cost just some ms in the file system, but would cost a multiple of that within a db (here UR), since then they would imply the (even repeated, in case!) use of the clipboard (or repeated sql accesses from the outside, and clipboard access to then process the results: a nightmare for the user!)

And that's why in a db where the user can't access the internals but from the outside - respective internal code would be much more elegant indeed -, construct b is much more preferable: quick navi / moves much less prone to annoying mistakes.


B)

As for the awful placement of the (very valuable otherwise!) AS Grid "Exclude" column, there is sub-key in Ultra Recall/Settings/SearchGrid, (i.e. not in the sub-key "Options"), "Order", with default value 0,1,2,3,4,5, so I have now tried to change that order to what I had intended, by changing the sub-key's value to 5,0,1,2,3,4, then rebooting, but that change does NOT work as hoped by me, the Exclude column remains last one in the grid, and the value has automatically (!) be reset to 0,1,2,3,4,5 - expected behavior? (Or should that have worked as I had hoped?) Would it some negligeable effort to put the Exclude column as first instead? (see above)


C)

Kyle, I'm very pleased you're looking into these problems, and it seems they are even more profound than you have already discovered, there will be some more glitches to be culled! ;-)

From what you have already found, it seems to me that my OR/AND problem started with 6.2 then (but without me making the connection since at the time, I hadn't already been in so fierce need for OR (and, to a lesser degree, the rest: ?/[])).

I had quickly reverted, at the time, to SearchNonAsciiTextNonCaseSensitive (SNA)=0, i.e. to the previous default, in order for the c1content column in table ftsitem_content (=also needed for plain text export, cf. our conversation in this forum at the time) bearing regular text).

I have now re-installed UR, v14 (i.e. replace v13), by just "overwriting" the previous installation (v13).

I have then checked the SNA sub-key, in order to change it to 0 again (from alleged value 1)...

and, it wasn't even there!

So I created it anew (DWord 32bit, with value 0: I know what I'm doing), then rebooted and checked the registry again: It was correctly there, and at the correct "abc" position.

I then opened UR (v14) and made the following tries - the BIG MOMENT indeed!

("contains keywords" is CK, "matches wildcard" is MW)

a OR b > works fine
a AND b > works fine

a b > (ok: implicit AND works as fine as it did before)
diesbezügliches > is found (quite rare word / word form, 4 correct finds), i.e. works fine

diesbez?gliches > (placeholder for the non-ascii-char) does NOT work (=no find)
> checked AS: switch from CK to MW is not done
> did the switch manually, then AS > NO find

diesbezüglic?es > (placeholder for the "h") NO find (instead of 4)
> checked AS, switch not made
> switch to MW made by me, then AS triggered > NO find (instead of 4)

es > 9720 matches (in order to check if the ? cut the search term into two, then just searched for the sub-string after the ?, but that does not seem to be the case

diesbezüglich?s > (placeholder for the "e") 90 WRONG "finds", but none of the 4 to-be-expected-ones (and no highlighting (which is "on") within those "finds"
> checked AS, switch not made (i.e. stayed at CK)
> switch to MW made by me, then triggered AS > NO find (instead of 4)

diesbezüglich* > (should include diesbezüglich, so even more than 4 finds expected) NO find
> checked AS, switch IS (!) done here (but as said, no find)
> triggered AS again, with that correct MW > NO find either

diesbezüglic[h]es > (correct "h" within []) no find
> checked AS, switch not done
> made switch to MW manually, then triggered AS > no find

diesbezüglic[ht]es > (since [] is meant to containr more than just one char) no find (instead of at least 4)
> checked AS, switch not done
> made switch to MW manually, then triggered AS > no find

I then checked the registry again: The SNA key was GONE!

Thus, it seems that when the SNA key is set to 0 (or anyway?), running UR will DELETE the SNA key altogether!

In theory, that could be also be done by something else in my system, but as said, after reboot, the key I had created, was there, then had disappeared after running UR, which might indicate UR itself does do the deletion?

Thus, the next step would be, it seems, to identify WHY the (sub-) key gets deleted, then re-do the search tries listed above, and check if any of them work better than; as said, AND and OR work fine now, but only those.

My next post will contain the UR Options (sub-) key (in its current state: v14, without the SNA key again); you might obviously want to delete it after copying the list.
Reply With Quote
  #4  
Old 04-09-2024, 06:22 AM
Spliff Spliff is offline
Registered User
 
Join Date: 04-07-2021
Posts: 207
Just the reg part here, see previous post.
Attached Files
File Type: zip ur.zip (2.6 KB, 146 views)
Reply With Quote
  #5  
Old 04-09-2024, 07:29 AM
kinook kinook is online now
Administrator
 
Join Date: 03-06-2001
Location: Colorado
Posts: 6,027
The only time UR removes registry values is when uninstalling and confirming the option to remove them, and then it removes all of them.
Reply With Quote
  #6  
Old 04-10-2024, 01:00 AM
Spliff Spliff is offline
Registered User
 
Join Date: 04-07-2021
Posts: 207
Registry = My mistake in part:
As said, I had re-installed UR, then searched for the SNA (sub-) key, first visually, then by RegEditor's search func > wasn't there;
I then had added the SNA key by hand, restarted the PC, checked: key was there.
I then had run UR, made the above search tries (with only OR and AND successful), and re-checked the registry, visually, I admit: key wasn't there anymore; then I posted the reg-dump and pretended the key (=which I had had to add before) wasn't there, but in fact, it's there, as the dump shows: at last position, with lots of non-abc reordering even in other parts, quite scrambled. Thus, the key definitely is there, for the time being.

So I continued to try QS:

diesbezügliches > 5 finds (checked: correctly highlighted)
(so 5 finds is the target number the other finds should met)

diesbez?gliches > 1 find
> checked AS: "contains keywords"
> I changed to MW > no find (5 expected)

diesbez[ük]gliches > no find (ditto for diesbez[ü]gliches)
> checked AS: "contains keywords"
> I changed to MW > no find (5 expected)

diesbezüglic?es > 1 find (checked AS: CK (MW expected?!); changed to MW: no find) (5 expected)

diesbezügliche? (= the ? for the end-s here, should NOT also find diesbezügliche since ? is deemed to stand in for exactly 1 char, not for 1 or 0) > 33 "finds", nothing highlighted; local searches then show that those "found" items contain the word diesbezügliche > the ? at the end obviously is discarded from the search

diesbezügliche > 53 finds (correctly highlighted and incl. the 5 occ of diesbezügliches)

diesbezügliche* > no find (* for 1, n or 0 chars, so * should be redundant here anyway)
checked AS: switch to MW is made (but as said, no find anyway, whilst 53 finds expected)

diesbez*gliches > no find
> checked AS: switch to MW is maid (but, as said, no find, with 5 finds expected)

without a non-ascii char:

dergleichen > 71 finds = ok

dergle?chen > no find
> checked AS: CK i.e. switch to MW is not made [edited for typo here]
> made the switch manually > no find (71 finds expected)

Thus, 2 problems:
- occurrence of (un-escaped) *,?,[] should trigger switch from CK to MW, but only * triggers that switch
- even with MW (manually, in AS), all of these placeholders are not correctly processed
(*-?-[]-problems obviously not linked to ascii-non-ascii)

Last edited by Spliff; 04-10-2024 at 01:07 AM.
Reply With Quote
  #7  
Old 04-10-2024, 09:25 PM
kinook kinook is online now
Administrator
 
Join Date: 03-06-2001
Location: Colorado
Posts: 6,027
I made some additional fixes (v6.3.0.16) for wildcard characters and contains keywords vs. matches wildcard search type and including NEAR/number in the non-lowercased FTS search expressions. Note that the wildcard syntax using [ ] and ? are not applicable to FTS search mode and only apply to SQLite non-FTS search.
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



All times are GMT -5. The time now is 02:46 PM.


Copyright © 1999-2023 Kinook Software, Inc.