Quick Search, Advanced Search: "OR" and (explicit) "AND" not working

[Spliff]

Is it some registry setting, intervening here for me, or is it a general problem? I have tried new UR versions, and versions back to April, 2021 (!), and they (don't) work all the same, in QS (Quick Search), in this respect:

a b
Finds items with a and b

a AND b
Only finds items which contain a, "and", and b, i.e. the "and" is resolved literally, not the Boolean way

a OR b
Ditto: Items must contain a, "or", and b, in order to be found

Thus, QS "AND" search is without problems since it's implicit, whilst QS "OR" search is impossible; deletion, and then re-build, of the full-text search features (which I have always "on") does not help.

My Search Options: General: Automatically no, Select no, Focus yes, Highlight yes yes; QS: Incremental 0, Max 0, Enable no, Perform no, Match no, and my search options in the Search dialog (5 check boxes): all 5 no.

Same problem for "Search titles only".

Very weird: Since my option "Highlight..." is ON, a and b are highlighted, but the respective "and" or "or" in the texts (which, as said, are needed then in order for the item to be found to begin with) are not highlighted.

From your "Quick Search" help item, I understand that Boolean search is just an option (sic!), from within those 5 options (= check boxes) within the Search dialog itself, and obviously, those 5 options do NOT also comprise an option "and/or resolved Boolean" (instead of literally);

thus, I assumed that, since AND is implicit anyway, the help item is outdated, and both AND (which would be redundant) and OR (which would be needed in order to override the implicit AND) were systematically resolved as Boolean codes, but that's obviously not the case, so is there some "toggle", somewhere (except for the fact that setting "and" and "or" to literal resolution would not make much sense anyway, I suppose)?

Also, you write, in that "QS" help item,

"Promote to Advanced Search: the Advanced >> button will convert your QS into an equivalent Advanced Search"

Thus, I did that, for some a OR b in QS; the Advanced Search appeared, with "Item" (i.e. not "Item Text") ... "contains keywords" "a OR b" ("a OR b" in the same field, entered automatically from my QS).

Again, with the 5 (partly different here) check boxes, and clicking on the "Start" button found nothing; ditto after me changing "Item" to "Item Text".

(Of course, my "a" and "b" are real words which are found separately, or with implicit "and".)

(Or is it a fault in my UR installation again? Otherwise, after my "totally complete" (?) UR re-install (cf. my "Repair" thread), UR seems to work without fault though...)


EDIT:

In the context of the other (, allegedly DLL damage) problem referred to, above , it should be noted that the above OR / AND problems also occur with new, totally "fresh" UR DBs (even "factory", within the .urd format, and with just 2 or 3 trial / "dummy" items), created from "File - New" (i.e. not by using some "template" UR db of mine, open or not within the "disaster" period. - On the other hand, I think I remember that before that, "OR" worked as expected (?; while implicit "AND" always works correctly, so I had never tried explicit "AND" before). Btw, my "Spelling Options" are "all no", my "Custom dictionary" being "Custom.dic" (never touched), and my "AutoCorrect Options" also being "all no" (i.e. check-boxes de-selected, and logically with no "Exceptions").

[kinook]

It's working as expected in my test.

Please send the info from https://www.kinook.com/Forum/showthread.php?t=3038

[Spliff]

So this is the last residual from my (allegedly DLL) disaster which I discovered on Christmas, even after a re-install which I had assumed to be "complete" finally; I'll further check the registry then.

[Spliff]

I can't resolve the problems I have with UR Search.

I de-installed UR by the UR de-install run, then rebooted and deleted any residuals I could find (Wise Care 365 and Voidtools Everything), then did a fresh UR install, left it at its factory settings, then tried my UR Search again, with the same (faulty / "no-") results as before. (I then only re-installed my UR settings: registry key, and the 4 UR settings files.)

I then installed UR on a second PC, with UR's factory settings (and left it at that), and I encounter strictly the same problems there; on that PC, AHK has never even been installed, let alone run, and it was just shortly connected to the internet 2 or 3 times, just in order to update Windows 10 from the MS site (and it was never connected to my "net" (LAN), not even to my printer, all software installs, except for W10 which came with the PC, by USB stick), and no problem whatsoever (except now for the UR Search), whilst my main PC had had some c: drive problems indeed (as described by me here, but no other problem with dozens of programs though, except for UR).

Thus it's obvious that my UR Search problems are not caused by problems on either of my PCs but by some setting or settings interaction; Tools - Options - Search is:
General: Automatically no (default is yes, same result), Select first no, Focus yes, Highlight no
Quick Search (QS): Incremental 0ms, Max 0
Then: Enable no, Perform no, Match no.
(Compact-and-Repair settings: Compact yes, Repair yes, Enable yes.)

Your help page "Matches Wildcard" (to be accessed by searching "matches wildcard" in the help's search field) states,
"The Matches Wildcard [Does Not Match Wildcard] comparison operator locates Info Items or Info Items with the specified Attribute that match the specified wildcard expression." - from my understanding as a non-native speaker, this means that ANY "wildcard operator" (i.e. plural) (i.e. ? or * or the regex [some] or the regex [^some], ditto for [some-some]) will trigger, when entered in QS, the respective "matches wildcard" AS (AdvancedSearch), whilst in all cases of the absence of these ?,*,[,] (or their "escaping" by [] or a second [/]), respectively) = in all "default cases", the QS will trigger the default "contains keywords" AS; a quick toggle between "Quick" and "Advanced" would confirm this switching between the two "Comparison" alternatives, i.e. would show that the "Comparison" column value / setting / choice in the AS "list" (always just one row here, for those QSs, "translated to AS") switched accordingly; your
"A Quick Search uses an (Item) Contains Keywords <keywords entered> type of criteria, unless one of the wildcard operators, or the " (double quote) symbol is included in the criteria, in which case a Matches Wildcard criteria will be used instead.", would confirm that I have correctly understood that somewhat not-too-evident "The Matches Wildcard [Does Not Match Wildcard] comparison operator" part: Any ? or * within the QS is understood as a wildcard operator, except when it's escaped by [] around it, and similar for [] when not escaped by doubling those.

Thus, the UR code analyzes the QS string, in order to decide which AS to apply, so far so good, and I verified by toggling from QS to AS; also, any space is "translated" as (implicit) AND, and any OR (surrounded by spaces) as (explicit) "OR".

Unfortunately, this "translation" from entered string to search command does not work correctly on both of my systems, but equally faulty:

1) (As already said above,) OR is not translated, but invariably considered as being literal (being written or or OR), so some OR other will neither find some nor other items, but only items comprising some AND or AND other; ditto (also said above) for explicit AND: some AND other will NOT find items with some AND other, but only items with some AND other AND and.

2) * is correctly translated, i.e. when the search string contains a *, then the AS switches from default "contains keywords" to "matches wildcard"; this is not also the case with everything else: AS is not switched accordingly, but remains at "contains keywords", with, as value, the literal string, e.g.:

A) a?b > contains keywords: a?1
B)(when I then, in AS, manually switch to "matches wildcard", and click on start, the AS does NOT find any ab1 or ac1 item, which are clearly there, but "0 matching items")

A) a[a-z]1 > contains keywords: a[a-z]1 > and no find, obviously
B) manual switch to "matches wildcard" > no find either, e.g. for ab1 (which exists)

etc. etc.

Thus, we have TWO / three problems here, at obviously TWO different positions within UR's code:

A) QS search string is not correctly translated to AS search (switch to "matches wildcard" is not made when necessary, i.e. when ? or [] BUT is made when * though; obviously, a one OR two would not need that switch to begin with):
* only is processed correctly

B2) Even the correct AS search then (triggered manually, with "Comparison" as "match wildcard") will not find any result since here again, ? and [] are interpreted literally, not as wildcard/regex codes
here again, * only is processed correctly

B1) Similar for correct AS search:
contains keywords: a1 OR a2 > the OR is processed literally, thus no search result


THUS, MORE CONCISE:

Two problems:

AND and OR systematically processed literally (in QS and in AS, both in lowercase and uppercase), so no search result

? and [] systematically processed literally (in QS and in AS), thus NO switch to "match wildcard" when necessary, and no search result even with "match wildcard" (just * processed correctly)

That, on old PC and new PC, with old UR and UR from March, 2024.

Obviously, within UR's code, there are glitches / interactions (persistent over the UR versions) which on two of my systems (1 perhaps to be considered "problematic", the other one certainly not) cause the above-described problems, even whilst obviously, on your system(s), they do not.



I now have de-installed (with the UR de-install program) UR from PC 2, incl. the user settings (so reverted to trial). I re-installed (newest) UR (default settings) on PC 2, and I created a NEW trial .urd (not using the UR default (i.e. "sample") .urd I had used to do my previous Search tries - never used UR otherwise on PC 2 anyway).

I created two notes, One and Two.

Screenshot 1 shows "One OR Two" not working in QS (screenshot shows the automatic translation to AS where NO switch to "matches wildcard" would have been necessary, but where none of the two to-be-expected search results shows up; ditto for the search "one OR two"; ditto (i.e. NO result) for the search "eins OR zwei", considering that in the "text" of the two items, I had put "eins" and "zwei", respectively (always without the double-quotes), but a screenshot showing those texts AND the (empty) search result at the same time does not seem to be possible.

Screenshot 2 shows "O?e" not working in QS (screenshot shows the automatic translation to AS where the necessary switch to "matches wildcard" has NOT been made, and the lack of result, whilst the "One" item should have been listed).

Since my UR installation of PC 2 is "factory", without any setting, or any possibility of components to be damaged, it seems there is a glitch for OR, (explicit) AND, ? and [] in UR, showing on SOME system, not on others (and unfortunately on both of mine), whilst implicit AND and * are correctly processed on any system.

(Needless to say that the "?" placeholder would be needed for "tags" in the form of e.g. xab where x is the tag code, a the value (here) for a first tag category and b the value (here) for a second tag category (and so on): searching for xa* works fine for "a" then "any", but ? would be needed for "any" then "b".)

(I also tried 2 items, with xxxa1 and xxxb1, respectively; I then searched for xxx?1 and for xxx[?]1 (considering that "literal vs. placeholder" for "?" might have been mixed up); for both, I got the same 5 (faulty) "xxx" results (since they just contain xxx), but none of the 2 needed ones, and, needless to say, the necessary switch from "contains keywords" to "matches wildcard" was not done.)

[Spliff]

To clarify:

NO "font" problem intervening here, I use originally installed Arial, in which my "?" and [] are the original ASCII / ANSI characters 63, 91, 93 (i.e. no fiddling with character substitution so that UR could not recognize my "special" ?[] anymore - just adding this info for "completeness").


NO "stop word" problem intervening here; I use "real" words, not just "a" and "b" and "one" and "two" and the like, but words that are correctly found when I just search for them, and which are also correctly found when I search for them with implicit (!) "AND", so

a > found

a b > (implicit "AND") is found when I put a and b into the same item

a AND b > (explicit "AND") is NOT found since "AND" treated literally (and of course, IS "found" when (!) ball three, a, b, and AND, are present in an item)

a OR b > as the previous "a AND b": treated literally, so "found" (only) IF item contains all three: a, or, and b

a?b > the "?" is treated literally, thus is "found" if item contains string "a?b"

just similar, not identical, for any [...]: some item with [[x]] >
[[x]] > "finds" any items with single "x" (AS translation: "contains keywords") but not the item with string "[[x]]"
> seems to indicate that string-analysis code is not perfectly discrete but ambiguous in some cases


4 work situations (all with fts "on"), faulty behavior identical (!) in all four (whilst amelioration had been expected from a) to any of the others, naturally):

a) PC 1, with internet, UR heavily used (many UR DBs), AHK running, lots of "services" running; explicit-AND, and OR, problems since many months, over several UR versions, but OR worked as expected, many months ago; "?" and "[]" problems just discovered recently i.e. never used, or tried to use them, before; W10 Pro english, latest version

b) as a), "clean" UR-re-install (version of March 8, 2024), but possibility of third-party "services" or whatever interfering

c) PC 2, no internet (except for rare W10 updates), perfectly "clean", just some programs installed, AHK never even installed (let alone used), almost never used (i.e. my "reserve PC"), (months-old) UR version installed but never used, never copied (or created) a UR db (onto) here; UR try just with 2, 3 newly created items within the default "sample" db; W10 Pro english, some version about 6 months old

d) as c), "clean" UR-re-install (version of March 8, 2024), UR try with newly created db with just 2, 3 items created (over the default items which are already present at start)


Identical problems:

QS (QuickSearch) > AS (Advanced Search):

QS: a OR b > no result
again QS: a OR b > AS > contains keywords a OR b (=correct) > no result > OR obviously treated literally

QS: a?b > no result (except for items containing literal string "a?b"
again QS: a?b > AS > contains keywords a?b (=necessary switch to "matches wildcard" NOT done)
> switch to "matches wildcard" done manually > no result either > even "matches wildcard" treats "?" (and []) literally, NOT as placeholders / wildcard chars


Hence two code problems in all 4 situations:

QS: occurrence of "?" (or [ and ]) in QS string does NOT switch the internal (= AS) processing "contains keywords" to "matches wildcard"

AS: any OR, AND, ?, [...] within AS "matches wildcard" are treated literally though (i.e. not only when "escaped" but all the time) (and additional problems with "[" escaping, not clearly identified by me)


In the other hand:

"OR", done by AS in 2 or more rows, does work as expected, i.e. AS
contains keyword a
OR contains keyword b
OR contains keyword c
etc
works correctly, whilst
contains keyword a OR b > does NOT work correctly (OR being treated literally)

"*" is correctly treated correctly, i.e. as wildcard, BOTH times:
* in QS string > switch, in AS, is made to "matches wildcard", and then,
in AS, the "some*" string is treated correctly, i.e. the * as wildcard > correct results


P.S.:

Obviously, given my previous problems with UR on PC 1 (situations a and b), I had expected that UR Search on PC 2 (situations c and d) would work correctly, and so I was ready to switch to PC 2 (=my "reserve" PC, as said) for that reason; I then was very surprised to encounter the same problems there (situation c), and then again, even after a new UR install (situation d); I now must expect that even on a brand-new PC (PC 3), which I would have to buy, the same problem either presents itself on-the-spot, or then will occur after some time of use; I can NOT re-install Windows on PC 2 since that would break my license for another, special, rarely-used but expensive, program, which is "bound" to the current "system" on that PC (I even lost 2 licenses on my PC 1, by a Windows re-install on there, some years ago...).

Would UR be another Delphi program? (= more or less the "standard" framework at the time of its creation) - With some Delphi DLLs or similar being damaged, under certain (but obviously not all) conditions, by Windows 10 updates? (and then not systematically updated, too, by UR re-installs?) ;-(

[Spliff]

EDIT:

1)

I got the idea to run UR from a USB stick, to PC 1, finally (it didn't work but after I had closed UR on PC 1 and ran it again from the stick then):

It opened the UR DBs I had last opened on PC 1; I QS-searched
a OR b
> it (almost*) correctly displayed the items which contained either a or b
*="almost" since it only failed to also list, as search results, two NEW items, one with a, the other one with b, created by me immediately before the search, i.e. editing the DB (on PC 1) "from" the stick, so this "almost" obviously is a "fts index not updated" problem: It seems that when running UR from a stick, the DBs should also be on the stick, not on the PC.

Thus, I again de-installed UR from PC 2 (on which I don't need it), again incl. "Delete customization data? YES" (= just as I had done before situation d) described above), then checked PC 2's registry for the UR key, in order to verify if before above-described situation d), the UR key had been deleted or not, so that the new UR install on PC 2 had re-created the registry key, or had just updated it: The key is completely gone, so also situation d) had been after a complete, "fresh" new UR re-install (and without copying the registry key or parts of it, or any of the 4 UR ini files from your list, or any parts of it).

Thus, situation d), AFTER a complete UR re-install, without (!) any settings beyond the default ones, AND showing the same problems as in situations a, b and c, seems to prove that NONE of my UR settings (i.e. be it in the reg key, or in any one of the 4 ini files), seems to cause the problem, and on PC 2, NO other program had been running in situation d).

Thus, I really don't know where to search for possible causes of this awful problem - and running UR from stick is very slow (but could be sped up somewhat by buying a better stick).

2)

Off-topic here: MyInfo / MyInfoApp (according to the "manual" available on their site) seems to do something what you do, filtering in the tree pane ("Data Explorer" in UR), but whilst you only allow for that filtering by flags, MI/MIA only allows it for text-within-title, so this text-in-title (i.e. not also within the content or in other elements) might also be a user's "tag"; thus it's obvious that their tree filtering is - I hate to say this! - FAR superior to UR's indeed, since IF the user knows about that limitation, they can put "textual core info" (tags) within the titles, then filter the tree by those:

- UR: 20 flags, MI: hundreds of different tags / sub-tags in case

- UR: 20 flags combinable by OR only (by clicking together them by AND in "Tools-Flags" > only ONE flag possible per item), MI: those hundreds of tags also combinable by AND (several (!) tags / title-sub-strings possible per item)

Thus, MI doesn't have "tree filtering by general search" either (and neither has RightNote, as far as I can see from their help file: trial reverts to free-version, and then you can't trial the "pro" version anymore, even many years later...), BUT by "search in titles" at least, which multiplies, as explained, the filter possibilities in comparison to UR at least.

(Also, refer to MI's "manual" (number 5.1 in there) in order to compare how they accept users' "search" input, and which is obviously the only really good way to do it... as for the search results' presentation, it's then not so brilliant indeed, and RightNote's are simply awful in comparison...)

It would be really a good thing to think about amending filtering, be it in tree or in search: My original assumption that it was more or less "standard", was very mistaken, BUT it IS standard in editors, grep tools: The original text's order is NOT scrambled by them, by/in filtering, and the additional milliseconds due to to the additional processing would be happily borne by the users: always as an option, the "regular" = faster representation being "tree-destructing", so it would be the user's decision to wait a little second on top, for MUCH better search results - also bearing in mind that the necessary RE-sort, BACK to tree-order, and AFTER the collection of the search results, could certainly not take THAT long, considering normal searches would bring just some, or some dozens of results, rarely hundreds, almost never more... and then, it would always be possible to include some code lines for, "if result_count > 500 then message 'Results not tree-sorted since too many' instead"... ;-)

[kinook]

UR is developed with Visual C++, and I don't see how a Windows update could affect searching within UR. The SQLite library is statically linked into UltraRecall.exe.

I'm still unable to reproduce the problem with AND and OR not working. Can you post the details from

https://www.kinook.com/Forum/showthread.php?t=3038

[Spliff]

My question re framework was because 25 years ago, .Net hadn't been there yet; I hadn't thought of MS tools before .Net; thanks for clarifying!

Sorry, my bad: The "stick version" (= situation e) = a fith one) is as faulty as the (newly and "from scratch") installed version, "new" items created on purpose weren't "found" since they didn't contain "or", as string, so "a or or", being processed in all 5 situations a)...e) as "a AND b AND or", wasn't found, obviously: after hours of trying, unsuccessfully, I then unfortunately tend to make even very stupid mistakes; a good night's sleep afterwards then clears the mind.

Btw, I tried with "dieses" and "jenes" (= "this" and "that", but certain to not introduce a "stop word" problem), and then, neither of the items just containing "dieses" OR "jenes" was listed as a "find", BUT when I then selected either of those two items manually, the "dieses" or "jenes" (in the text = content) was highlighted (which is only expected for search results, not also for the first item the user navigates to, after a search), whatever it was: if I selected the "dieses" item after the search (title = "v1"), "dieses" was highlighted, and if I selected the "jenes" item after the search (title = "v2"), "jenes" was highlighted, and vice versa: highlighted "search find" (but not also appearing within the search results) for the very first item selected after the search, whatever it was.

For a "dies?s" search, or for a "jen?s" search, I get some search results indeed, but with nothing being highlighted, since the whole content of any of those "hits" doesn't contain any string which could be translated to "dies[a-z]s" or "jen[a-z]s", ditto for searches for "dies[a-z]s" > "contains keywords" > LONG "hit" lists which don't make any sense, without any hit, neither placeholder- nor literally-wise, and then, changed to "matches wildcard" manually, no hits anymore - all this with fresh installs, not only after my "adjusting" the UR settings to my wishes.

The next step would obviously be to buy a third W10/11 PC.

(As said, OR works fine by AS in 2 independent rows, and I would have needed the ?-placeholder for "tag-pivoting": sometagA[a-z] being possible by searching for sometagA, but for searching / filtering by sometag[a-z]A I would have needed a "working" ?-placeholder, and, obviously, such "compound" tags would be (=for other UR users; would have been, for me) extremely useful for FILTERING-by-searching.

I then thought I could replace them by TWO tags instead, then searching = filtering for them by implicit AND:
sometag* someothertagA [1]
or
someothertagA sometag* [2]
= instead of
uniquetag?A
that is, and the * assuring the switch to "matches wildcard" will be made (it is made indeed).

Since 1 didn't work, I tried 2 (i.e. the * at the end), and that didn't work either, i.e. correct translation to AS, but no result, and:

When, in AS, I then click on the "Quick" button, I see that the QS search string has been enclosed in double quotes, so it reads now
"someothertagA sometag*"
(for both versions, 1 and 2, i.e. the * being "within" the string, or at the end of the string, does NOT have any implication).

Thus, it's obvious that the code which translates the search string, makes false assumptions, caused by the * here, or then caused by the translation to "matches wildcard", itself (correctly) caused by the *). (When I search for "sometagA someothertagB", it's found, and back to QS, it's without the "", but of course, we're in, and stay in, "contains keywords" then.)

Thus, even * does not work correctly BUT when it's only one single search string, with the * at its end:
sometagA* [that works indeed]

When the two tags are in the same line of the content, i.e. not in different lines, then
sometagA someothertag*
is successful ("matches wildcard" is done, and item is found), but back to QS, it now reads
"sometagA someothertag*" [= phrase search]

whilst (logically then)
someothertag* sometagA
doesn't find the item (which contains sometagA and someothertagB)
since, whilst AS shows, matches wildcard someothertag* sometagA
back-to-QS button brings
"someothertag* sometagA"
adding (unwanted) phrase search to placeholder/ wildcard use again, but possibly treating the * literally here because it's not at the very end of the (as said, unwanted) phrase search;

on the other hand, when I put
someothertag* sometagA
into the item content, into the same line = as "phrase", and expecting that "since" UR treats the * literally, that "phrase" should be found then, AS
matches wildcard someothertag* sometagA
does NOT find anything, albeit, back to QS, the search now reads,
"someothertag* sometagA"
where clicking on "Start" brings "0 matching items", and back to AS, it now reads
matches wildcard "someothertag* sometagA"
and back to QS, it now reads
""someothertag* sometagA""
and with this back-and-forth, I can multiply the "" at will, whilst it has never been intended as a phrase item to begin with!


I hope you can replicate (parts of?) my "a * brings unwanted double quotes (but doesn't treat the * literally anyway, and treats it as placeholder just at the very end of the forced phrase)" problem (which seems to indicate the search string might be processed faultily, internally)?

(Obviously, I closely verified my tag strings, and my search strings, in order to not make any mistake here.)

[kinook]

Again, please provide the info from

https://www.kinook.com/Forum/showthread.php?t=3038

[Spliff]

Since I only know my 5 systems (PC 1 before clean re-install and after*, PC 2 before clean re-install and after*, stick*; *=here at least, the thread-3038 data obviously is the "factory" one, cf. the "Trial" in the above screenshots' caption), I can't speak for other UR users' systems here, so with this reservation:

Before, control-end systematically (?) went to the very LAST row-AND-column (= "Value", so to the very last value of them all), then "typing" (even by macro, i.e. without previous F2) started to change the "Value"'s string value; that's why in the other thread, I suggested to put string values within the very LAST row of Advanced Search.

Now, with your implementation of my suggestion of some Discard / "Exclude" (yours is the better wording indeed) column, whilst I had had in mind "Exclude" as the very FIRST column (i.e. even before "Relationship"; but without specifying that, I admit), you added that column as very last one, so quick "value" input is excluded now, the user or macro have to fiddle around with navigation within the AS rows / columns, beyond a simple control-end, and, worse,

sometimes, the focus in the AS grid is STUCK within "Item/Attribute" drop-down, and then, NO key navigation whatsoever (and with any modifier key) will LEAVE that drop-down, and, for example, both "up" and "down", AND "left" and "right" navigate up and down within that drop-down's entries, whilst at least "left" and "right" would be expected to LEAVE that drop-down, and set focus to "Relationship" or "No", respectively, i.e. go to the adjacent column. (From one AS to the next, sometimes, the new focus is where it was placed on leaving the previous AS, but more often than not, it's re-set on the Item/Attribute column, and then, the errance within the drop-down starts...

This makes AS macros unreliable since the macro "thinks" it's in the grid's intended target column/row, while in fact erring within the Item/Attribute drop-down, and the user has no other way than to use their mouse, in order to navigate off that drop-down.


As for (and always speaking of my 5 systems only) the "Vapor-Func" (OR, *, ?, [...]) described above, to resume:

Even * is unreliable, sometimes it works, but triggering the switch AND phrase search, then again not, seems to depend not only on the place of the * but also on the search terms, so it's not usable, so none of them all work:

For OR searches, I need multiple rows in AS (with those new additional problems described here; I'll have to go back to a UR version before even Feb, 2024 (since not even the March version, but also the Feb version already have the "Exclude" column, complicating things);

ditto for tag searches, only xab (with x meaning, "it's a tag", a being the category, b being the value) works, but works indeed;

as said, implicit AND works fine, so I can put several such tags into the items, or rather into the titles, since that way, I always see them, i.e. know any item's tags without searching, and then some xab xch xgj (even Quick) search (implicit AND) will work, whilst for xab OR xac OR xah, I need an AS grid with 3 rows, so I make several ASs, with 2 OR rows, 3 OR rows, etc., and IF I hold it all within the very same indentation level, even tree order is preserved, so that can I work from within the search results, as I would otherwise do from within the Data Explorer (which becomes unrealistic by experience, considering that filtering (in tree order) is necessary, but that, given the fact that any item just can bear one flag, but several tags, whilst Data Explorer filtering is just possible by flags, not also by tags: thus, working from within the search result grid is necessary BUT doable indeed - just the practical use of the AS grid, again and again, doesn't come really that handy, to put it as "light" as it gets.

MyInfo(App)'s search input field, together with its what you might call "scope codes", and with its (expected, and working) parentheses to group when necessary, obviously are the way to go, since that's the nearest thing to the target SQL "select" code, AND slight changes / variations (which are complicated fiddling around with the AS grid in UR in many cases) are done almost immediately, be it manually by the user, or then by some macro input into that input field.

I admit that UR's input field, together with working OR/?/[...] (i.e. if those ain't Vapor-Funcs as in my 5 systems), is not that bad to begin with, but leaves out any "more-than-just-one-scope" which then need the AS grid... which can't be regarded as "user-friendly" as it is.

This is NOT to advertise MI/MIA here - its search results do NOT respect tree order (and miss most of UR's sort variants) -, so it's questionable if MI's better search input (!) will really be so much more valuable in the end, given its search output (!), but its "tree filter" (by strings = tag or any other word-start-string), is infinitely more valuable than UR's "tree filter" (by flags).

Thus, copying that functionality from the competitor would certainly be UR's most important enhancement in years, and seems to be doable:

Tree order (as in UR for tree filtering) is respected (obviously)
Find-as-you-type, i.e. "live" display of the results
space is "AND"
NO */?/etc
NO sub-strings, just word-start-strings (but not necessarily title-start-strings, and btw: otherwise no implicit AND would be possible), i.e. x finds xa, xar, etc., but e.g. ar will not find xar

JUST for titles, and that makes it possibly very much easier than with "strings-from-everywhere", since the titles are listed in traditional sql tables (i.e. no fts problems), can be indexed accordingly, and UR currently IS able to respect the tree order, by filtering by flags, here in the tree, so the same should be possible with title string bits.

AND, as said before, it should be done in a flat list, but without discarding deeper-down items (i.e. parent items may, or may not, fulfill the condition(s).


Information management being about information retrieval in the end, the most perfect retrieval possible should be the object.

And yes, the ostentatious "lack" of interest (or better: so they insist on simulating) into all these matters, from UR co-users, is so much de-motivating - why should they be given anything then they don't pay extra for, right?! - that I can only qualify it as "appalling" indeed... which spares me to have to use that nasty "pearls before ..." expression, yeah.


ADD-ON:

To clarify: 20 flags seems "not too bad", the number 20 divided by 20 "values"... of the SAME category, but once you divide that number by several (!) categories (!), i.e. color, italics, bold, underline, then combine those (e.g. bold instead of regular for one category, different values then by different colors; another category by italics, combined with the aforementioned categories, by regular/bold, by color for the category represented by colors, and so on), you find yourself with "almost nothing", and then you have to renounce on categories, just have enough possible values-in-all, i.e. 20, to combine two categories, one of the two with a strict minimum of possible values; thus, for categories asking for many values, or even for sub-categories, you are forced to resort to tagging by text-tags... and that will set you back to Search, forces you to LEAVE the "Data Explorer", and then, Search's functionality's problems get into your, the user's way... (That being said, just formatting plus colors for filtering bear an inherent number limit, different formats not being available in number, and colors not being discernable in big numbers by the eye... whilst (additional) text-tags (as explained above) don't come with such bounds but with virtually no number restrictions; thus for more detailed filtering, beyond very basic filtering for just one category, they are the better solution to begin with, or in other words, sharply multiplying the number of flags instead wouldn't have been but a pis-aller anyway. Oh, and I should have made this clear indeed: MI's tree filtering by text-tags (=as many such tags as you need), whilst being much better than UR's tree filtering by flags (just ONE flag possible per item, and just up to 20 all combinations combined), because of its lacking "OR" (at least), isn't that formidable in the end, so "just copying" the MI func couldn't then, unfortunately, be considered "state of the art" yet, and I think it's better that I say so myself, instead of blatantly inviting to some, "and when that's done, you'll continue to complain", unavoidable otherwise:

At the end of the day, working on a really good Search functionality would be so much more worthwhile / beneficial:
- (Alternative) Input by input field, with e.g.: t:some for title, c:some for content, e:some or just some for everywhere, and so on (=scope), parentheses just like in SQL (i.e. the SQL target string anyway)
- Tree preservation upon demand (= by option) and, in case, by re-ordering after the search if it's really that much more complicated to do from search start - from the NON-observation of tree order, within the search results from MI, RN (RightNote) AND UR, I infer that all PIMs have adopted the Adjacency List Model in its most basic flavor (whilst UR's items' titles are stored in three different locations, though: so much for strict redundancy avoidance), and that makes me think that even with some minor ALM enhancement (lots of such ALM-to-"Common Table" extension examples being available in the web), i.e. one with not-too-much additional "write" tasks, already could be some BIG help in refining the Search?

Also, let's face it: On the "Mac" side, tools with "Search Results with the (first-per-item) relevant paragraph" are available; RN offers "n words (!) before and/or m words behind the search term", which makes the results more or less unreadable, since the n before and/or the m before is always much too few or much too many, and I have not found any word wrap, so, and...

to share at least immediately constructive idea here - goodwill! - UR users are invited to put - and, if necessary, even to copy! - the core info (incl. text-tags if they don't prefer to put them systematically into the titles) of any relevant (!) item into the very first paragraph, then browsing through the search results (by up and down) will present that core info much faster, than by having to then, one-by-one for (almost) every "find", SCROLL to the core parts, or rather, those parts where the search string(s) are to be found; I suppose it might be possible though to introduce automatic scroll up to the first "find" in UR instead (i.e. screen display would start with the start of the paragraph which contains the "find"), upon option (i.e. of just highlighting the finds)? (Alternatively, the user might use a big screen, with full-height content pane, AND and large search results pane on its side, so as to hopefully see the relevant parts even without having them on top and/or having to scroll down to them - the success of this strategy obviously depends on the user's average item text length...)

And the user might also display the "Item Text" column, since then, a mouse-over will - fast! - display the "full" text (i.e. as far as there is room for it) within some fly-over pane, albeit in a quite tiny font size, and without formatting, the latter being perfectly understandable since obviously, that "quick-n-dirty" text preview is retrieved directly from the sql table column... which then should make it even less helpful when you store the plaintext of your items in all-lowercase (writing from memory here, not having tried out recently) - be that as it might, font resizing by user setting should be easy to implement, to make this goody finally worthwhile, but many user will probably not even have discovered it in its current form? So they're informed now, but as said, the kind / lack of "interaction" with UR co-users here makes it ultra-difficult to consider them "fellow UR users", which obviously would be the ideal.


Oh, and - OT here - I "upped" my "move-to" script by (after the obligatory ^x, obviously, then) first inserting a {down}, then only have it make the navigation to the target(s) (if there are intermediate targets, too, you can count those, then adjust the number of necessary "Go - Go back"s accordingly), then do the ^v, and then do (at least one) GoBack (shortcut at your leisure), and, heureka, you can do other moves from where you left before the current one; the problem here being that this just works if you do the moves one-by-one, even when they go to the very same target, OR if at least the very last one of the items you will have selected for moving, is not immediately situated above another item, to be moved (and thus, if you move some "block", pay attention that you select the "bottom" item as last one), since otherwise, your "GoBack"(s) will stay within the target area, not go back to the "area of origin" (similar if you send an {up} instead of the down-key) - such being the sudden problems you encounter once you try to put some "outliner" to "power use"... (And, think on it: even knowing the number of the selected items - which can programmatically be retrieved indeed: it's an element of "window text" in case -, will not help since the {down} (similar for {up}) will not work from the visually-"last" item, but from the last-selected one, and if that's within a block, contiguous or not, you'll end up "somewhere"; ditto for copying items, obviously.) - In other words: From the "outside", "only so much"* can be done, at the end of the day, whilst constant optimization of code can "do it all", and this example is just a blatantly simple one... (*=meaning here: if you remember to select the "bottom" of several selected item last, it works as expected, except of course if you work from end-of-tree, but then, a simple {end} after the move will do) - You see here that co-users might have gotten lots of additional goodies if they had made just and strictly even minimal efforts onto what you might call "collaboration" (and just another example: within "global text replace", you must insert some "trick" at some position of recurring part of the script, and then, global replace "from the outside" works fine, but nada... not even one single "thank you", let alone adding / sharing your ideas, hints: nada: Third-party UR users, you call that "constructive"? Wanna make me laugh? ;-) (And when then, on top of that, I pile up lots of problems with the software I can't resolve, for my own needs, I'm more than just fed-up... understandably, I might dare say.)

[cnewtonne]

Spliff ...

What I have found to be most effective way of reporting issues is to ...

1. Use a numbered list of reproduction steps vs. long paragraphs & pages.
2. Describe steps using the UI command words. Use same terms the app is using.
3. Use screenshots and recordings if you want to show reproduction steps/errors.

In this thread, you have described a usability issue using 6,526 words. Using default MS Word settings, these are 13 pages! Still, it is hard to understand what you're describing. I thought I understood you from the very first post, but you lost me later.

Software is science. As such, it is concise and is described in exact steps and words. Make it easy for yourself and your fellow users by adhering to these simple steps, please!

Thank you very much.

[Spliff]

( QS/AS being QuickSearch / Advanced Search, respectively, and obviously: )

A)

In practice, it's NOT possible to "flatten" the hierarchy in UR, since:

a) compare this construct:

'a...
____.b...
________Something:

.b has 'a (or more precisely: \'a) in its lineage, and
Something has both \'a... and \.b... in its lineage, and
Something, and also anything within the possible sub-tree of Something, can be found by QS (triggered from 'a... or from .b) Something (with "LS" checked), "LS" being the checkbox "Limit search to selected item in Data Explorer pane", in other words, "Local search", i.e. "just in the current sub-tree";

b) with this construct:

.a - ...
.ab - ...
____Something:

even .ab (which corresponds to above's 'a .b, obviously) does NOT have .a (or then, \.a) in its lineage, all the less so would Something have \.a in its lineage, thus:
triggering, from .a, an "LS" QS search, for .ab - ... or any of .ab -...'s child items (including Something, and any of Something's child items, and so on), would fail; you would have to do a global search instead, triggering lots of unwanted "finds", in case.

Obvious solution to this problem: You build an AS instead (if the * works on your system that is), with:
- item - contains keywords - Something (or whatever string the "target" item may contain, in its title, content, "Notes", etc., anywhere below .ab in general, or below Something in particular)
- AND - Lineage - matches wildcard - \a* (or \ab* if you want to do an even more specific search)

Thus, with strict technical (=indentation-level) hierarchy only, you can avoid AS, whilst with just a "virtual" hierarchy", .a and .a[a-z] being placed onto the same indentation level, you will need the AS... which is a nightmare if you do perhaps hundreds of "searches" (i.e. "greps", filterings) a day (which is my case).

B)

But this "flattening out" the hierarchy in UR would be of interest indeed, considering that:

Whenever you want to [/B]navigate[/B] to some "item" (i.e. some "inter-title", prefixed with some "marker", as the ' and . used above), it's evident that a simple entry .a, or then .ar will be oh so much more "easy" - if those "."-entries are immediately below your source item, i.e. in level 1, than to navigate first to some entry 'a, then to its child item .b (without knowing if item 'a has already been expanded or not)... and then, if you are mistaken of the existence / naming of an item .b "within" parent range 'a, and if some other range 'whatever is expanded at the same time, your macro will "target" the .b... inter-title within, let's say, the 'c... parent range, since your macro will use the in-built tree-navigation, by "targeting" the "very next" .b... item, be that within its intended range 'a..., or then within the 'c... range if that one is expanded, and some .b... item is existant within that range...

And yes, you can then, i.e. "after the facts", trigger a "UR command line" retrieval, and then check if your ".b..." find (or further find below .b...) has a \'a within its lineage, but that costs 1,000 or 1,500 ms more (since your macro will have to retrieve, by clipboard, that "UR command line" first, before being able to analyse it), and this procedere cannot be called "elegant" or "smooth" in any way...

C)

On the other hand, within the file system, such macros are without problems, since in both situations, a) and b), you just work with * (i.e. a*\b*\*/something or then, ab*\*/something) and thus you have free choice between the paradigms "deep" or "flat", and both procedures work fine, by any professional code: I do both, i.e. all four, both in AHK and in Everything, the * being a placeholder for some sub-routine in case but which, on a modern PC, just takes some ms indeed, and NO markers like ' or . being needed, since the \ will lead the code, whilst in a db like UR, the LEVEL is ambiguous, i.c. can not be counted fast by the number of \ within the path, and then, the (navigational, move, copy, link) TARGET is specified by that path, and reached immediately, whilst in a db like UR, by scripting from the outside, it will have to be reached, step-by-step, and time-consuming path verifications being necessary at every such step then.

D)

Thus, yes, in DBs like UR, and especially in such DBs, a "flattened" hierarchy is largely preferable (since it does away with the need for "command line" verification, "is the target .b... really a child item of intermediate target 'a...?", all these not being needed within a file system navigation / move / etc), but, frankly spoken, UR's AS "costs" you several seconds "on top" of what would have been necessary, for every single search, and thus, if there was available some ENHANCED search box, as in MI and other software, and which would allow for entering
l:a* Something [l: = "within lineage", or better: p: for "path"]
or then, if you want it to be more precise:
l:ab* Something

this would not only speed up searching / filtering enormously, but it would also spare you to first navigate to the "parent item" in question, then check the "LS" (see above), but you could trigger this QS (as you could trigger the respective, but enormously time-consuming AS now) from anywhere, and a search "from current item", i.e. within the sub-hierarchy of that current item, could be triggered by:
l: Something
i.e. without placing any value behind the l:

And similar for any other QS:

Kyle, could you please consider such an enhanced Search Box, but with publishing a list of all its codes you consider to implement / use in it, in order for allowing us to have a look upon them, and to give our view?

And yes, such a system would sacrify the possibility for searching for some:some, but then, nobody really would need to search for some:some, except if they had used that string-form for tags, and why couldn't they replace those tags then by some.some indeed?

And yes, the some before the : should be as short as possible (hence my suggestion to publish them here, before implementation, or then, why not invite us to make such list, open to suggestions then, from the current "Item/Attribute" drop-down):

pt:some (always without any space around the ":") would be parent-title:some, and
pt: would be parent-title:current-item's-title, as any empty value after the : would be "current-value" wherever it makes sense.

The necessary "core" code for such a UR Search Revolution had already been implemented, by your transcription from AS Search grid to SQL string, and, frankly, UR's current AS Search grid is a nightmare, costing an additional 5s even when applying a script, and costing minutes for manual AS search: the absence of a FAST entry for the variable part(s) of those "stored searches" being the main problem, the "grouping by indentation instead of by parentheses" being the secondary one...

and the good news is, twofold:
- user's parentheses within the enhanced search box could be transferred 1-by-1 to the sql code (i.e. no additional coding work needed), and
- the current AS grid could be preserved without any additional effort, for people who prefer that, well, "very original" (and very time-consuming), way of searching. ;-)

[kinook]

Regarding the original topic about AND and OR searches not working, I did find a bug. There is logic to support case-insensitive Unicode searching (see SearchNonAsciiTextNonCaseSensitive), which is enabled by default, and this logic lowercases the indexed text and search terms that are entered, and it was incorrectly lowercasing AND and OR in the search phrase, which prevented the FTS search engine from treating these as AND and OR search clauses. This has been fixed in the latest download (v6.3.0.14). A workaround is to disable that registry setting if you don't need case insensitive Unicode search capability.

Regarding the other topics discussed here, can you boil this down to plain English? I am not quite following what you wrote.

Note: Updated in v6.3.0.15 to also handle NOT and NEAR commands.

[Spliff]

A)

I hope my previous post is understandable, though, when detailing the problems which arise with navigation in level hierarchy (equivalent to the file system's folder hierarchy, and in which the user doesn't encounter those problems, so that in order to do RELIABLE navigation (which is even more important for moving / copying items), a flattened (or you might call it virtual) hierarchy is preferable.

Btw and inadvertently, I had left out the main argument for that flattened hierarchy (construct b instead of construct a):

Construct a)

'a (all these expanded currently, or even just to be expanded by navi-macro)
__.a
____be it some item here
__.b (the .marker assures you get here, instead of the "b" item before)*
__.d (they can be grouped by subject, no abc-sorting needed)
__etc
''b
__.a
__.c

Now, the user wants to "go" to 'a .d:
> (s)he will enter ad, the macro does rest, and (s)he gets to the .b "in" 'a
> perfect

(And you wanted to go to the "b..." item, you entered aab instead, the macro navigating to 'a..., then to .a..., then to b...)

But the user is mistaken, wants to go to 'a... .d... in fact, but thinks that .d... starts with c ((s)he will have named the .d... by some synonym .c... but fails to remember):
> enters ac, expects to get to 'a .c (but that doesn't exist)
> boom: will arrive at 'b .c instead: Not too much harm for simple navigation, but really awful for moves! (Given the fact that all such navigation is done by character entry within the Data Explorer.)

Now the same situations in construct b):
.a
.aa
__be it some item here
.ab
.ad
.aetc
.b
.ba
.bc

You enter a, the macro goes to .a; you enter ab, the macro goes to .ab; you enter aab, the macro goes to .aa, then to the b-item (child of .aa).

But now, if the user is mistaken, thinks .ad is called .ac, (s)he enters ac or acsomething in order to get even deeper into .ac - and (s)he will NOT "land" somewhere in .b but in .a (since .ac doesn't exist): You "land" (or move something as last item into) the parent range, where you (or the moved item(s) will be much more "at home" than you, or they, will be somewhere within .b.

In the file system, that's no problem, since any path element's (=intermediate folder's) existence will be checked, and there is does not exist, the macro stops, and goes to (or puts into) the nearest parent folder (which does exist): These verifications cost just some ms in the file system, but would cost a multiple of that within a db (here UR), since then they would imply the (even repeated, in case!) use of the clipboard (or repeated sql accesses from the outside, and clipboard access to then process the results: a nightmare for the user!)

And that's why in a db where the user can't access the internals but from the outside - respective internal code would be much more elegant indeed -, construct b is much more preferable: quick navi / moves much less prone to annoying mistakes.


B)

As for the awful placement of the (very valuable otherwise!) AS Grid "Exclude" column, there is sub-key in Ultra Recall/Settings/SearchGrid, (i.e. not in the sub-key "Options"), "Order", with default value 0,1,2,3,4,5, so I have now tried to change that order to what I had intended, by changing the sub-key's value to 5,0,1,2,3,4, then rebooting, but that change does NOT work as hoped by me, the Exclude column remains last one in the grid, and the value has automatically (!) be reset to 0,1,2,3,4,5 - expected behavior? (Or should that have worked as I had hoped?) Would it some negligeable effort to put the Exclude column as first instead? (see above)


C)

Kyle, I'm very pleased you're looking into these problems, and it seems they are even more profound than you have already discovered, there will be some more glitches to be culled! ;-)

From what you have already found, it seems to me that my OR/AND problem started with 6.2 then (but without me making the connection since at the time, I hadn't already been in so fierce need for OR (and, to a lesser degree, the rest: ?/[])).

I had quickly reverted, at the time, to SearchNonAsciiTextNonCaseSensitive (SNA)=0, i.e. to the previous default, in order for the c1content column in table ftsitem_content (=also needed for plain text export, cf. our conversation in this forum at the time) bearing regular text).

I have now re-installed UR, v14 (i.e. replace v13), by just "overwriting" the previous installation (v13).

I have then checked the SNA sub-key, in order to change it to 0 again (from alleged value 1)...

and, it wasn't even there!

So I created it anew (DWord 32bit, with value 0: I know what I'm doing), then rebooted and checked the registry again: It was correctly there, and at the correct "abc" position.

I then opened UR (v14) and made the following tries - the BIG MOMENT indeed!

("contains keywords" is CK, "matches wildcard" is MW)

a OR b > works fine
a AND b > works fine

a b > (ok: implicit AND works as fine as it did before)
diesbezügliches > is found (quite rare word / word form, 4 correct finds), i.e. works fine

diesbez?gliches > (placeholder for the non-ascii-char) does NOT work (=no find)
> checked AS: switch from CK to MW is not done
> did the switch manually, then AS > NO find

diesbezüglic?es > (placeholder for the "h") NO find (instead of 4)
> checked AS, switch not made
> switch to MW made by me, then AS triggered > NO find (instead of 4)

es > 9720 matches (in order to check if the ? cut the search term into two, then just searched for the sub-string after the ?, but that does not seem to be the case

diesbezüglich?s > (placeholder for the "e") 90 WRONG "finds", but none of the 4 to-be-expected-ones (and no highlighting (which is "on") within those "finds"
> checked AS, switch not made (i.e. stayed at CK)
> switch to MW made by me, then triggered AS > NO find (instead of 4)

diesbezüglich* > (should include diesbezüglich, so even more than 4 finds expected) NO find
> checked AS, switch IS (!) done here (but as said, no find)
> triggered AS again, with that correct MW > NO find either

diesbezüglic[h]es > (correct "h" within []) no find
> checked AS, switch not done
> made switch to MW manually, then triggered AS > no find

diesbezüglic[ht]es > (since [] is meant to containr more than just one char) no find (instead of at least 4)
> checked AS, switch not done
> made switch to MW manually, then triggered AS > no find

I then checked the registry again: The SNA key was GONE!

Thus, it seems that when the SNA key is set to 0 (or anyway?), running UR will DELETE the SNA key altogether!

In theory, that could be also be done by something else in my system, but as said, after reboot, the key I had created, was there, then had disappeared after running UR, which might indicate UR itself does do the deletion?

Thus, the next step would be, it seems, to identify WHY the (sub-) key gets deleted, then re-do the search tries listed above, and check if any of them work better than; as said, AND and OR work fine now, but only those.

My next post will contain the UR Options (sub-) key (in its current state: v14, without the SNA key again); you might obviously want to delete it after copying the list.

[Spliff]

Just the reg part here, see previous post.