Quick Search, Advanced Search: "OR" and (explicit) "AND" not working

[Spliff]

Is it some registry setting, intervening here for me, or is it a general problem? I have tried new UR versions, and versions back to April, 2021 (!), and they (don't) work all the same, in QS (Quick Search), in this respect:

a b
Finds items with a and b

a AND b
Only finds items which contain a, "and", and b, i.e. the "and" is resolved literally, not the Boolean way

a OR b
Ditto: Items must contain a, "or", and b, in order to be found

Thus, QS "AND" search is without problems since it's implicit, whilst QS "OR" search is impossible; deletion, and then re-build, of the full-text search features (which I have always "on") does not help.

My Search Options: General: Automatically no, Select no, Focus yes, Highlight yes yes; QS: Incremental 0, Max 0, Enable no, Perform no, Match no, and my search options in the Search dialog (5 check boxes): all 5 no.

Same problem for "Search titles only".

Very weird: Since my option "Highlight..." is ON, a and b are highlighted, but the respective "and" or "or" in the texts (which, as said, are needed then in order for the item to be found to begin with) are not highlighted.

From your "Quick Search" help item, I understand that Boolean search is just an option (sic!), from within those 5 options (= check boxes) within the Search dialog itself, and obviously, those 5 options do NOT also comprise an option "and/or resolved Boolean" (instead of literally);

thus, I assumed that, since AND is implicit anyway, the help item is outdated, and both AND (which would be redundant) and OR (which would be needed in order to override the implicit AND) were systematically resolved as Boolean codes, but that's obviously not the case, so is there some "toggle", somewhere (except for the fact that setting "and" and "or" to literal resolution would not make much sense anyway, I suppose)?

Also, you write, in that "QS" help item,

"Promote to Advanced Search: the Advanced >> button will convert your QS into an equivalent Advanced Search"

Thus, I did that, for some a OR b in QS; the Advanced Search appeared, with "Item" (i.e. not "Item Text") ... "contains keywords" "a OR b" ("a OR b" in the same field, entered automatically from my QS).

Again, with the 5 (partly different here) check boxes, and clicking on the "Start" button found nothing; ditto after me changing "Item" to "Item Text".

(Of course, my "a" and "b" are real words which are found separately, or with implicit "and".)

(Or is it a fault in my UR installation again? Otherwise, after my "totally complete" (?) UR re-install (cf. my "Repair" thread), UR seems to work without fault though...)


EDIT:

In the context of the other (, allegedly DLL damage) problem referred to, above , it should be noted that the above OR / AND problems also occur with new, totally "fresh" UR DBs (even "factory", within the .urd format, and with just 2 or 3 trial / "dummy" items), created from "File - New" (i.e. not by using some "template" UR db of mine, open or not within the "disaster" period. - On the other hand, I think I remember that before that, "OR" worked as expected (?; while implicit "AND" always works correctly, so I had never tried explicit "AND" before). Btw, my "Spelling Options" are "all no", my "Custom dictionary" being "Custom.dic" (never touched), and my "AutoCorrect Options" also being "all no" (i.e. check-boxes de-selected, and logically with no "Exceptions").

[kinook]

It's working as expected in my test.

Please send the info from https://www.kinook.com/Forum/showthread.php?t=3038

[Spliff]

So this is the last residual from my (allegedly DLL) disaster which I discovered on Christmas, even after a re-install which I had assumed to be "complete" finally; I'll further check the registry then.

[Spliff]

I can't resolve the problems I have with UR Search.

I de-installed UR by the UR de-install run, then rebooted and deleted any residuals I could find (Wise Care 365 and Voidtools Everything), then did a fresh UR install, left it at its factory settings, then tried my UR Search again, with the same (faulty / "no-") results as before. (I then only re-installed my UR settings: registry key, and the 4 UR settings files.)

I then installed UR on a second PC, with UR's factory settings (and left it at that), and I encounter strictly the same problems there; on that PC, AHK has never even been installed, let alone run, and it was just shortly connected to the internet 2 or 3 times, just in order to update Windows 10 from the MS site (and it was never connected to my "net" (LAN), not even to my printer, all software installs, except for W10 which came with the PC, by USB stick), and no problem whatsoever (except now for the UR Search), whilst my main PC had had some c: drive problems indeed (as described by me here, but no other problem with dozens of programs though, except for UR).

Thus it's obvious that my UR Search problems are not caused by problems on either of my PCs but by some setting or settings interaction; Tools - Options - Search is:
General: Automatically no (default is yes, same result), Select first no, Focus yes, Highlight no
Quick Search (QS): Incremental 0ms, Max 0
Then: Enable no, Perform no, Match no.
(Compact-and-Repair settings: Compact yes, Repair yes, Enable yes.)

Your help page "Matches Wildcard" (to be accessed by searching "matches wildcard" in the help's search field) states,
"The Matches Wildcard [Does Not Match Wildcard] comparison operator locates Info Items or Info Items with the specified Attribute that match the specified wildcard expression." - from my understanding as a non-native speaker, this means that ANY "wildcard operator" (i.e. plural) (i.e. ? or * or the regex [some] or the regex [^some], ditto for [some-some]) will trigger, when entered in QS, the respective "matches wildcard" AS (AdvancedSearch), whilst in all cases of the absence of these ?,*,[,] (or their "escaping" by [] or a second [/]), respectively) = in all "default cases", the QS will trigger the default "contains keywords" AS; a quick toggle between "Quick" and "Advanced" would confirm this switching between the two "Comparison" alternatives, i.e. would show that the "Comparison" column value / setting / choice in the AS "list" (always just one row here, for those QSs, "translated to AS") switched accordingly; your
"A Quick Search uses an (Item) Contains Keywords <keywords entered> type of criteria, unless one of the wildcard operators, or the " (double quote) symbol is included in the criteria, in which case a Matches Wildcard criteria will be used instead.", would confirm that I have correctly understood that somewhat not-too-evident "The Matches Wildcard [Does Not Match Wildcard] comparison operator" part: Any ? or * within the QS is understood as a wildcard operator, except when it's escaped by [] around it, and similar for [] when not escaped by doubling those.

Thus, the UR code analyzes the QS string, in order to decide which AS to apply, so far so good, and I verified by toggling from QS to AS; also, any space is "translated" as (implicit) AND, and any OR (surrounded by spaces) as (explicit) "OR".

Unfortunately, this "translation" from entered string to search command does not work correctly on both of my systems, but equally faulty:

1) (As already said above,) OR is not translated, but invariably considered as being literal (being written or or OR), so some OR other will neither find some nor other items, but only items comprising some AND or AND other; ditto (also said above) for explicit AND: some AND other will NOT find items with some AND other, but only items with some AND other AND and.

2) * is correctly translated, i.e. when the search string contains a *, then the AS switches from default "contains keywords" to "matches wildcard"; this is not also the case with everything else: AS is not switched accordingly, but remains at "contains keywords", with, as value, the literal string, e.g.:

A) a?b > contains keywords: a?1
B)(when I then, in AS, manually switch to "matches wildcard", and click on start, the AS does NOT find any ab1 or ac1 item, which are clearly there, but "0 matching items")

A) a[a-z]1 > contains keywords: a[a-z]1 > and no find, obviously
B) manual switch to "matches wildcard" > no find either, e.g. for ab1 (which exists)

etc. etc.

Thus, we have TWO / three problems here, at obviously TWO different positions within UR's code:

A) QS search string is not correctly translated to AS search (switch to "matches wildcard" is not made when necessary, i.e. when ? or [] BUT is made when * though; obviously, a one OR two would not need that switch to begin with):
* only is processed correctly

B2) Even the correct AS search then (triggered manually, with "Comparison" as "match wildcard") will not find any result since here again, ? and [] are interpreted literally, not as wildcard/regex codes
here again, * only is processed correctly

B1) Similar for correct AS search:
contains keywords: a1 OR a2 > the OR is processed literally, thus no search result


THUS, MORE CONCISE:

Two problems:

AND and OR systematically processed literally (in QS and in AS, both in lowercase and uppercase), so no search result

? and [] systematically processed literally (in QS and in AS), thus NO switch to "match wildcard" when necessary, and no search result even with "match wildcard" (just * processed correctly)

That, on old PC and new PC, with old UR and UR from March, 2024.

Obviously, within UR's code, there are glitches / interactions (persistent over the UR versions) which on two of my systems (1 perhaps to be considered "problematic", the other one certainly not) cause the above-described problems, even whilst obviously, on your system(s), they do not.



I now have de-installed (with the UR de-install program) UR from PC 2, incl. the user settings (so reverted to trial). I re-installed (newest) UR (default settings) on PC 2, and I created a NEW trial .urd (not using the UR default (i.e. "sample") .urd I had used to do my previous Search tries - never used UR otherwise on PC 2 anyway).

I created two notes, One and Two.

Screenshot 1 shows "One OR Two" not working in QS (screenshot shows the automatic translation to AS where NO switch to "matches wildcard" would have been necessary, but where none of the two to-be-expected search results shows up; ditto for the search "one OR two"; ditto (i.e. NO result) for the search "eins OR zwei", considering that in the "text" of the two items, I had put "eins" and "zwei", respectively (always without the double-quotes), but a screenshot showing those texts AND the (empty) search result at the same time does not seem to be possible.

Screenshot 2 shows "O?e" not working in QS (screenshot shows the automatic translation to AS where the necessary switch to "matches wildcard" has NOT been made, and the lack of result, whilst the "One" item should have been listed).

Since my UR installation of PC 2 is "factory", without any setting, or any possibility of components to be damaged, it seems there is a glitch for OR, (explicit) AND, ? and [] in UR, showing on SOME system, not on others (and unfortunately on both of mine), whilst implicit AND and * are correctly processed on any system.

(Needless to say that the "?" placeholder would be needed for "tags" in the form of e.g. xab where x is the tag code, a the value (here) for a first tag category and b the value (here) for a second tag category (and so on): searching for xa* works fine for "a" then "any", but ? would be needed for "any" then "b".)

(I also tried 2 items, with xxxa1 and xxxb1, respectively; I then searched for xxx?1 and for xxx[?]1 (considering that "literal vs. placeholder" for "?" might have been mixed up); for both, I got the same 5 (faulty) "xxx" results (since they just contain xxx), but none of the 2 needed ones, and, needless to say, the necessary switch from "contains keywords" to "matches wildcard" was not done.)

[Spliff]

To clarify:

NO "font" problem intervening here, I use originally installed Arial, in which my "?" and [] are the original ASCII / ANSI characters 63, 91, 93 (i.e. no fiddling with character substitution so that UR could not recognize my "special" ?[] anymore - just adding this info for "completeness").


NO "stop word" problem intervening here; I use "real" words, not just "a" and "b" and "one" and "two" and the like, but words that are correctly found when I just search for them, and which are also correctly found when I search for them with implicit (!) "AND", so

a > found

a b > (implicit "AND") is found when I put a and b into the same item

a AND b > (explicit "AND") is NOT found since "AND" treated literally (and of course, IS "found" when (!) ball three, a, b, and AND, are present in an item)

a OR b > as the previous "a AND b": treated literally, so "found" (only) IF item contains all three: a, or, and b

a?b > the "?" is treated literally, thus is "found" if item contains string "a?b"

just similar, not identical, for any [...]: some item with [[x]] >
[[x]] > "finds" any items with single "x" (AS translation: "contains keywords") but not the item with string "[[x]]"
> seems to indicate that string-analysis code is not perfectly discrete but ambiguous in some cases


4 work situations (all with fts "on"), faulty behavior identical (!) in all four (whilst amelioration had been expected from a) to any of the others, naturally):

a) PC 1, with internet, UR heavily used (many UR DBs), AHK running, lots of "services" running; explicit-AND, and OR, problems since many months, over several UR versions, but OR worked as expected, many months ago; "?" and "[]" problems just discovered recently i.e. never used, or tried to use them, before; W10 Pro english, latest version

b) as a), "clean" UR-re-install (version of March 8, 2024), but possibility of third-party "services" or whatever interfering

c) PC 2, no internet (except for rare W10 updates), perfectly "clean", just some programs installed, AHK never even installed (let alone used), almost never used (i.e. my "reserve PC"), (months-old) UR version installed but never used, never copied (or created) a UR db (onto) here; UR try just with 2, 3 newly created items within the default "sample" db; W10 Pro english, some version about 6 months old

d) as c), "clean" UR-re-install (version of March 8, 2024), UR try with newly created db with just 2, 3 items created (over the default items which are already present at start)


Identical problems:

QS (QuickSearch) > AS (Advanced Search):

QS: a OR b > no result
again QS: a OR b > AS > contains keywords a OR b (=correct) > no result > OR obviously treated literally

QS: a?b > no result (except for items containing literal string "a?b"
again QS: a?b > AS > contains keywords a?b (=necessary switch to "matches wildcard" NOT done)
> switch to "matches wildcard" done manually > no result either > even "matches wildcard" treats "?" (and []) literally, NOT as placeholders / wildcard chars


Hence two code problems in all 4 situations:

QS: occurrence of "?" (or [ and ]) in QS string does NOT switch the internal (= AS) processing "contains keywords" to "matches wildcard"

AS: any OR, AND, ?, [...] within AS "matches wildcard" are treated literally though (i.e. not only when "escaped" but all the time) (and additional problems with "[" escaping, not clearly identified by me)


In the other hand:

"OR", done by AS in 2 or more rows, does work as expected, i.e. AS
contains keyword a
OR contains keyword b
OR contains keyword c
etc
works correctly, whilst
contains keyword a OR b > does NOT work correctly (OR being treated literally)

"*" is correctly treated correctly, i.e. as wildcard, BOTH times:
* in QS string > switch, in AS, is made to "matches wildcard", and then,
in AS, the "some*" string is treated correctly, i.e. the * as wildcard > correct results


P.S.:

Obviously, given my previous problems with UR on PC 1 (situations a and b), I had expected that UR Search on PC 2 (situations c and d) would work correctly, and so I was ready to switch to PC 2 (=my "reserve" PC, as said) for that reason; I then was very surprised to encounter the same problems there (situation c), and then again, even after a new UR install (situation d); I now must expect that even on a brand-new PC (PC 3), which I would have to buy, the same problem either presents itself on-the-spot, or then will occur after some time of use; I can NOT re-install Windows on PC 2 since that would break my license for another, special, rarely-used but expensive, program, which is "bound" to the current "system" on that PC (I even lost 2 licenses on my PC 1, by a Windows re-install on there, some years ago...).

Would UR be another Delphi program? (= more or less the "standard" framework at the time of its creation) - With some Delphi DLLs or similar being damaged, under certain (but obviously not all) conditions, by Windows 10 updates? (and then not systematically updated, too, by UR re-installs?) ;-(

[Spliff]

EDIT:

1)

I got the idea to run UR from a USB stick, to PC 1, finally (it didn't work but after I had closed UR on PC 1 and ran it again from the stick then):

It opened the UR DBs I had last opened on PC 1; I QS-searched
a OR b
> it (almost*) correctly displayed the items which contained either a or b
*="almost" since it only failed to also list, as search results, two NEW items, one with a, the other one with b, created by me immediately before the search, i.e. editing the DB (on PC 1) "from" the stick, so this "almost" obviously is a "fts index not updated" problem: It seems that when running UR from a stick, the DBs should also be on the stick, not on the PC.

Thus, I again de-installed UR from PC 2 (on which I don't need it), again incl. "Delete customization data? YES" (= just as I had done before situation d) described above), then checked PC 2's registry for the UR key, in order to verify if before above-described situation d), the UR key had been deleted or not, so that the new UR install on PC 2 had re-created the registry key, or had just updated it: The key is completely gone, so also situation d) had been after a complete, "fresh" new UR re-install (and without copying the registry key or parts of it, or any of the 4 UR ini files from your list, or any parts of it).

Thus, situation d), AFTER a complete UR re-install, without (!) any settings beyond the default ones, AND showing the same problems as in situations a, b and c, seems to prove that NONE of my UR settings (i.e. be it in the reg key, or in any one of the 4 ini files), seems to cause the problem, and on PC 2, NO other program had been running in situation d).

Thus, I really don't know where to search for possible causes of this awful problem - and running UR from stick is very slow (but could be sped up somewhat by buying a better stick).

2)

Off-topic here: MyInfo / MyInfoApp (according to the "manual" available on their site) seems to do something what you do, filtering in the tree pane ("Data Explorer" in UR), but whilst you only allow for that filtering by flags, MI/MIA only allows it for text-within-title, so this text-in-title (i.e. not also within the content or in other elements) might also be a user's "tag"; thus it's obvious that their tree filtering is - I hate to say this! - FAR superior to UR's indeed, since IF the user knows about that limitation, they can put "textual core info" (tags) within the titles, then filter the tree by those:

- UR: 20 flags, MI: hundreds of different tags / sub-tags in case

- UR: 20 flags combinable by OR only (by clicking together them by AND in "Tools-Flags" > only ONE flag possible per item), MI: those hundreds of tags also combinable by AND (several (!) tags / title-sub-strings possible per item)

Thus, MI doesn't have "tree filtering by general search" either (and neither has RightNote, as far as I can see from their help file: trial reverts to free-version, and then you can't trial the "pro" version anymore, even many years later...), BUT by "search in titles" at least, which multiplies, as explained, the filter possibilities in comparison to UR at least.

(Also, refer to MI's "manual" (number 5.1 in there) in order to compare how they accept users' "search" input, and which is obviously the only really good way to do it... as for the search results' presentation, it's then not so brilliant indeed, and RightNote's are simply awful in comparison...)

It would be really a good thing to think about amending filtering, be it in tree or in search: My original assumption that it was more or less "standard", was very mistaken, BUT it IS standard in editors, grep tools: The original text's order is NOT scrambled by them, by/in filtering, and the additional milliseconds due to to the additional processing would be happily borne by the users: always as an option, the "regular" = faster representation being "tree-destructing", so it would be the user's decision to wait a little second on top, for MUCH better search results - also bearing in mind that the necessary RE-sort, BACK to tree-order, and AFTER the collection of the search results, could certainly not take THAT long, considering normal searches would bring just some, or some dozens of results, rarely hundreds, almost never more... and then, it would always be possible to include some code lines for, "if result_count > 500 then message 'Results not tree-sorted since too many' instead"... ;-)

[kinook]

UR is developed with Visual C++, and I don't see how a Windows update could affect searching within UR. The SQLite library is statically linked into UltraRecall.exe.

I'm still unable to reproduce the problem with AND and OR not working. Can you post the details from

https://www.kinook.com/Forum/showthread.php?t=3038