Unofficial plugin / extension system design notes

[teratorn]

Here is a first draft at design notes for a Plugin / Extension system for Ultra
Recall. This is just technical, but there should also be thought put in to an
Online Directory, Forums, and Rating Systems for Scripts (produced by Kinook or
by 3rd parties).

You might want to allow automated script installation... Basically what you do
is just register a new File Extension and Mime Type in Windows, and then
provide download links to files that are served with that extension and mime
type. They then are automatically opened by UR (and after checking the
CRYPTOGRAPHIC SIGNATURE, they are allowed to be installed).

Otherwise it will show a Big Bad Warning before allowing the installation of
untrusted plugins.

So anyway... Here goes:

Do *not* try to be language agnostic. Pick the language that you want to use,
and then USE IT FULLY. Don't artificially handicap any of the power that the
language, and the STANDARD LIBRARY OF THAT LANGUAGE (this is key!) has to
offer.

-= DATABASE RELATED APIs =-

DB QUERY API. There should be an API that allows the user to QUERY the database
for sequences of ITEMS, just in the same way that the Advanced Search dialog
box allows. DO NOT PROVIDE DIRECT ACCESS TO SQL. You don't want to restrict
your freedom to change the DB schema in the future, and we want scripts to be
forward-compatible as much as possible.

Items should be returned as objects that have members that represent the
fields. Or perhaps there will simply be one member (e.g. "fields") that is a
Dictionary (associative array) that maps Field Names to Field Values. Field
Vales should be Strong Typed, meaning a Date field ought to have some
language-specific DateTime object that represents those values.

To change DB VALUES you then will either directly MUTATE (modify) a Value
Object that you already hold a reference to, OR (In the case of Immutable Value
Types) you create a new value, and insert it on the Item Object.

TRANSACTION SUPPORT. Provide a simple wrapper for the SQLite transaction
START/END calls, so that scripts can do their work inside nice transactions,
yay!

(Advanced Feature) SINGLE-COLUMN, or MULTI-COLUMN QUERY API. Provide an Query
API that allows the programmer to pull only certain columns out of the DB.

Say you only can what the Flag value is for the items that match your query...
The result of the query would then NOT be a sequence of Object Items, but
rather a sequence Flag Values (of whatever type the Flag is represented by).

If you specify more than one column, then the result would be a sequence of
sequences (tuples), with each sub-sequence containing the Values of the Columns
for that particular Item.

(Advanced Feature) LAZY QUERY ITERATION. Instead of returning full in-memory
sequences, you can instead return Iterable Objects (how you do this depends on
the language being used, of course).

ITEM ID QUERY. Every item in the database should have a unique, global ID (in a
particular .urd file) that can be used to retrieve it.

I'm borrowing a lot of these concepts from the excellent Axiom database
available for Python (it uses SQLite v. 3 under the hood)

-= THREADING CONCERNS =-

Without looking at your codebase I can't know what kind of threading model you
utilize in UltraRecall. So it's pretty hard for me to say what kind of concerns
you will have to deal with in keeping scripts Thread Safe.

My basic thoughts are that all Scripting APIs (where ever possible) should do
locking implicitly and automatically where needed. Script Programs should be
allowed to spawn multiple threads, because sometimes you have to use threads to
do certain things.

However, I don't personally like writing multi-threaded programs very much.
It's extremely difficult to do it well, and to not introduce horrible,
impossible to debug errors in to your programs.

That said, I would lean towards providing an asynchronous Reactor pattern
implementation (event loop) that operates inside a single thread, and that
multiple independent Scripts can use at the same time. This is the same
event-driven model used in GUIs.

If you use Python as the extension language, then Twisted is the obvious choice
for this. You don't have to do any work here, and you give access to a
tremendous amount of power. (I'm a committer on Twisted, so I'm biased, but it
really is the best... :)

-= DATA MODEL =-

So, you ask, how do you represent the wide variety of content that UR can
handle? Good question.

I don't know.

Lets take Rich Text as an example. I have no idea how you represent Item
Content internally (as a Microsoft RTF document???), so I'm not sure what you
will have to do to expose that content in a programmable fashion.

Ideally, Scripts would be able to manipulate Rich Content in all of its detail.

You may need to invent an XML Schema here, that you can map your internal
representation on to. (X)HTML, anyone?

Or if you could somehow use MS Word internally, then you can use the Word 2003
XML format, while though it is ugly, it is certainly usable and extremely
feature-rich.

So if you use XML, or something else, you will need to provide a DOM API,
similar in nature to the DOM APIs provided in Javascript and by various XML
parsers.

If you use XML, then you can just grab one of the open source DOMs for your
language of choice.

Now, of course, The Tree in UltraRecall is fundamentally a DOM, and you may
wish to provide some integration here, and use the same API for accessing the
ULTRA RECALL DOM, and the various CONTENT DOMs.

Maybe it call all be part of a single DOM (one big tree)?

I'm just thinking out loud here.. and without knowing anything about the
codebase I'm not sure what the best way forward would be...

The DOMs I'm talking about above are fundamentally TEXT BASED. But that may not
be entirely suitable for UR. You may want to provide a DOM that contains not
just generic NODES, but Strongly Typed Objects (that can contain Strongly Typed
Children Objects and Strongly Typed Attributes).

-= GUI ACCESS =-

So, scripts obviously need to be able to perform any actions that the user can
perform through using the various menu items, shortcut keys, and buttons that
are in the GUI. The script should also have access to the various Controls that
fundamentally make up the UR GUI.

E.g. The Script API should allow access to the various Panes, and it should be
able to e.g. Make them wider.

I don't talk here about *modifiying* anything that is *shown* in a pane,
because that fundamentally belongs to the DB / DATA MODEL APIs. There should
obviously be an API to read which Item is currently Selected in the Current
Tab. And thus you can change the data on that Item, and it will reflect in the
various Panes.

An API to add new Tabs, enumerate and remove existing Tabs, and so on.

Access to the clipboard ( though you could just let people use native OS-level
access to the clipboard) instead of providing your own Scripting API for it.

And so on...

How hard all this is, and how it may be done of course depends on how you have
written your GUI layers. Is it just straight MFC/win32 API?

Scripts also need to be able to create their own GUI Controls and have them be
integrated with the Standard Controls where needed. E.g. A Script should be
able to provide a new toolbar, and you could then place, and customize that
toolbar like any of the built-in ones.

Scripts also need to be able to create their own Form Dialogs and Popup
Windows, etc.

You don't really need to do anything special here except choose a language that
is powerful enough to access Windows APIs, and then Scripts can do anything
they need to do.

Some thought will need to go in to how, and at what level, GUI integration is
allowed between Controls created by the Script and those created by UR.

-= NETWORKING =-

Network access is huge. E.g. I want a Plugin that lets be enter a Movie Name (
I have a database of all the DVDs in my collection), and it then pulls down all
the metadata for that movie from IMDB in to the new Movie Item that I am
inserting.

Or I might write a plugin that syncs my Contacts with Gmail.

As above, just give access to a language that already has all this. (Twisted is
the most powerful way to do networking under Python, bar none :)

-= DEPENDENCIES =-

There ought to be some simple mechanism that one Script can use to tell UR that
it depends on another script. E.g. The script GmailSync might depend on
MetaScript >= 2.0. MetaScript could be a Script that provides various library
routines that are useful to multipe Scripts, and it may provide some mechanisms
to allow inter-script communication (over loopback sockets or Windows Pipes,
etc). Or it could also provide a simple mechanism for a given Script to
enumerate other loaded Scripts and get references to their public objects,
allowing direct communication and collaboration between scripts.

-= CONCLUSION =-

So basically I want to be able to do EVERYTHING. But I know that I won't get
that in the end... That's OK. Even a small fraction of these things, exposed
through a powerful language, would be very useful.

The word "Script" is kind of misleading... I probably should have used the word
Plugin or Extension above.

The very first thing you guys could do would be to allow direct SQLite access
to ENCRYPTED database files. There are plenty of things I could do on my own if
I just had that. Of course it's not pretty, and it's not very safe, and it's
not gonna be forward-compatible. But I'll take it!

I'll take, and be grateful for (and pay for!) anything you give us. But of
course, the more the better!

P.S. Thanks for taking the time to read this. I spent quite a lot of time
thinking and writing, however it is still only a first draft, and I look
forward to constructive comments.

[teratorn]

-= HOOKS / EVENTS =-

The Plugin API should also provide a variety of Events that the Script author can "Hook" in to. E.g. there will be Events for the following things:

* Ultra Recall Startup
* Database Opened
* Database Closed
* Command executed (either view menu item, button or key combination)
* Item Focus Changed
* Item Moved / Deleted
* Item Created
* Item Copied / Linked
* Item Attribute / Content Changed
* Reminder Fired Off
* Tab Created / Closed / Moved
* ...

And so on...it should be fairly intuative what things are most interesting to
provide hooks for.

In the Script language you will simply define callbacks that accept various
parameters related to the Event being fired. The Objects, Items or Controls
that are passed to the callback will depend on the particular Event in question.

The hook is created when the Script uses an appropriate API to register its
callbacks with the Events it is interested in. It should be able to unregister
hooks, too, of course.

[teratorn]

-= LIVE / DYNAMIC SAVED SEARCHES =-

It would be very useful if Scripts could "subscribe" to a "feed" of content, so to
speak. This concept can be implemented by hooking a saved search, then a
callback is fired whenever things are added or removed from the results of
those saved searches.

This implies they need to be updated each time the state of the database
changes in such a way as to have the possibility of modifying the saved search
results. I imagine there are various optimizations one could perform to
accomplish these semantics without constatly running full queries against the
DB.

[quant]

Originally posted by teratorn
-= LIVE / DYNAMIC SAVED SEARCHES =-
This implies they need to be updated each time the state of the database
changes in such a way as to have the possibility of modifying the saved search
results. I imagine there are various optimizations one could perform to
accomplish these semantics without constatly running full queries against the
DB.
why would they do it? Too complicated ... and it would mean that UR could become sluggish when doing basic insert/delete because the software would need to check all the searches whether they should be updated. And I think there is already some search caching in place ...

[teratorn]

Originally posted by quant
why would they do it? Too complicated ... and it would mean
that UR could become sluggish when doing basic insert/delete
because the software would need to check all the searches
whether they should be updated. And I think there is already some search caching in place ...

It's just an idea. If it's too slow or too complicated to implement
then I'm sure Kinook will figure that out.

Regarding its utility, a couple examples should suffice: A Script
could subscribe to an "Upcoming Events" saved search and fire
off an email when new items appear in that Virtual Folder
(another name for saved searches). Or you could use this to
implement calculations, or items that provide statistics, or a
summary view of matching items... e.g. a Script could add up all
the Mileage and Work Actual fields for matching items and input
that information in to some kind of summary or report (it's always
up-to-date because the script is triggered each time the saved
search changes)

This is just more simple than hooking *all* object creation and
then inspecting each object to see if it matches your criteria.

And it's more powerful because we allow the *user* to *define*
the selection criteria using the Ultra Recall UI which they are
already familar with. No fuss, no muss.

I may have made a mistake by describing this in a separate
section... It's basically the same thing as hooking item
creation/deletion for a given *folder*, it's just that in this case it's
a Saved Search (Virtual Folder), but nonetheless it is just another
node in the tree with (virtual) children. At least, that is how I look
at it.

[teratorn]

I mentioned Word 2003 XML as a possible format to represent and expose Rich
Content to scripts, etc.

It's actually not a very good suggestion... Office Open XML (OOXML) is the wave
of the future. This is Microsoft's new international standard (ISO) supported by
Office 2007 (with support for Office 2000 and up provided by a downloadable
extension).

OOXML is used for Word documents but also for Presentations and
Spreadsheets.

WordprocessingML is the actual XML dialect used for rich Word document
content.

The other contender (and ahead of OOXML, in many ways) is Open Document
(ODF). This format is already experiencing wide acceptance, and Microsoft has
pledged full native support for it in Office 2007 SP2, to be released shortly.

While ODF is superior in almost every way, and I would prefer to use it in any
project that I do, it may have some downsides for use in Ultra Recall.. this is a
question for further research to answer.

Anyway, could I get more feedback on this entire thread? What do you all
think?

[glenviewjeff]

Thanks for putting this together. Is Kinook considering this? I have been itching to develop plugins for UR since day-one. It sure would help UR evolve much more quickly than he can accomplish alone.

Jeff

[kinook]

There doesn't seem to be much interest in it, and most feature requests do not appear to be amenable to end-user implementation, so I would say it isn't highly likely, but we remain open to considering adding a basic plug-in/extension capability. It would be very helpful to hear some specific ways users would intend to utilize such a capability. Thanks.

[teratorn]

Originally posted by kinook
There doesn't seem to be much interest in it

It's not *generally* interesting, no. Such is programming - However the cost you pay by not taking full advantage of the aspirations and energy of enthusiastic users such as myself, quint, glenviewjeff, and so on is steep. I've been a user since July '07... that's a long time during which I would have contributed greatly to the UR Plugin ecosystem, and I'm just one guy. The power is in the networking effects.
and most feature requests do not appear to be amenable to end-user implementation, so I would say it isn't highly likely,
I'm not sure what "amenable to end-user implementation" means, exactly... are you saying it's too hard to put a nice UI on top of all this stuff? Because it certainly would be.. but in my view, you don't *need* to have nice UIs on anything except for the Plugin Search function and the Plugin Rating System.. Crypto verification etc. Just the most basic parts need to be "nice". Developers will handle the rest just fine.
but we remain open to considering adding a basic plug-in/extension capability.
Yes, please :)
It would be very helpful to hear some specific ways users would intend to utilize such a capability. Thanks.
A few things that I would do right away:

- Plugin to sync contacts directly w/ Gmail.

- Plugin to integrate gmail *email* itself directly in UR, in the style of Outlook integration, but bypassing it entirely. Outlook is off-limits for the security conscious user (imnsho ;)

- Plugin to sync Info items with Trac Wiki pages, and also with Plone... Trac is HUGE and used everywhere.

- Web server that serves out various parts of my tree as html/css/javascript.

- Alert monitoring that sends emails and SMSs to my phone for important events. Because I use UR in a virtual machine it is easy to miss reminder popups. Would also just integrate the alerts with my Host's "tray" (Ubuntu has a nice system-wide alerting feature)

- Fast sharing/edting of an info database between two UR instances over Internet + simple users/groups system. (I have friends *and* business colleagues that I've put on to UR... but I can't share my DBs effectively.)

These are all about *communication* and *collaboration*... the two area where UR is severely lacking, imho. Thanks for listening. -teratorn

[quant]

Originally posted by teratorn
- Plugin to integrate gmail *email* itself directly in UR, in the style of Outlook integration, but bypassing it entirely. Outlook is off-limits for the security conscious user (imnsho ;)
security, gmail, you made my day :)

[teratorn]

Originally posted by quant
security, gmail, you made my day :)

Hehe, thanks

One thing here I want to add to the discussion... I think it's important that we don't start trying to overthink the
"problems" here... it strikes me that I've thrown together a LOT of information in the thread, because I
was reaching for the stars.

So, I've been sitting here for the last 10 minutes thinking of the simplest, most straight-forward thing
that Kinook could do to start unlocking peoples' Info Databases to the power of a programming language,
and thereby to the pool of talent that already lies within the user community - a pool of talent that will most
certainly grow as a result of definite steps to facilitate it. The fruits will be slow in coming, or so it may seem at
times, but that shouldn't stop anyone from sowing such well-conceived seeds.

So, A definite proposal:

- The information contained in an Info Database.. the real meat & potatoes of the content we collect and
create needs to be accessible and modifiable by Plugins.

- Plugins must be distributed as source code packages... no precompiled binaries. The only people I
trust to run binaries from is Kinook, and nobody is adding to my list here without code review being
possible :)

- Design and implement a Data Access Layer that gives read-write access to nodes in the tree, and various
other bits of data that fit here, but don't actually live in the public tree (Settings, perhaps? read-only?). This is
the BIG ONE. I've already talked above about this layer, so please refer to those notes, and also keep clearly in
mind that we *want* to program according to the "Ultra Recall model" that we are all *already* familiar with...
e.g. the data query API should work just like the advanced search window. Perhaps Kinook has already
implemented such a data layer internally? Sounds like a likely guess.

- Utilize the Python programming language for its ease of learning, wide-spread adoption, huge user
community, and it's professional programming power (not to metion it's free and open-source).


(Python is written entirely in C (with much of the standard library and utilities being written in Python
itself. This makes it attractive to UR since UR is written in C++. Python provides both C and C++ APIs. Kinook
would need to author an "ultra_recall" C "extension".. this process is documented. That extension would
implement the interface to UR's internal API and data structures, and expose them via nice Python APIs).


- Utilize a "setuptools"-based solution for Plugin packaging and distribution... almost all the challenges
that Kinook would face in managing dependencies between Python-based plugins can be solved by using
this as a starting point... we simply need to ensure that any security concerns are addressed, and that we
aren't coding ourselves in to a corner in any way.

- New "Plugin Security" attribute that can be set on any node in the tree.. values of "No Scripting Access", "Read-
Only", "Read-Write". The setting is recursive to child nodes. This is an easy way to "protect" data from any
access by Plugins... this peace of mind goes a long way for me... I don't want Plugins to be messing around
behind my back.. much better to just expose those parts of my tree explicitly.. or if I prefer, for some DBs, I
can set this attribute on the root node. Yay.

- People can start posting Plugins to a special forum up in here... to get started with... trust will be vested in the
author's reputation and in reviews/comments and the fact that anyone can read the source code to the plugin.


That's really it... SQLite + Ultra Recall + Data Layer + Python == FUN.

Consider this an offer of assistance... I'm happy to answer questions via email whereever I could lend my
expertise... I've been writing Python since 1999, and I have a lot of professional coding experience. But maybe
this is better done through separate blogs, or a shared Trac instance, or some other public forum (this forum is
not really ameanable to "semi-internal" discussion between developers). I wish to invest more heavily in
my own usage of UR, and I'm starting to put a lot of other people on to it as well... so doing this kind of
work is just a good investment in my future.

Let the good times roll... ;)

[kinook]

Based on your examples, it sounds like what you really want is a high-level UR-specific API for reading+writing the database (rather than the lower-level SQLite interface), not a full-blown extension system for the UR GUI itself (ala Word VBA or Firefox). That would certainly be more feasible and would make it possible for users (and potentially simpler for us) to greatly increase UR's interoperability with the outside world. It's definitely food for thought. Thanks.

[glenviewjeff]

Yes, database access would be a very nice start. I saw the post on another thread about how to access the database, but was scared off by the strong warnings.
I'm not particular about what programming language the API is--I'm even willing to accept a command-line interface to the database.
As far as applications go, I want to be able to create a plug-ins to do:
- Syncing in and out of ListPro so I can have some of my UR database available on my PocketPC.
- Create a web-based “dashboard” that I can access from anywhere that shows UR items of interest
- Integration with Launchy (application launcher) to index UR items for instant retrieval
- Enter multiple UR items (one per line) with single paste of clipboard text
- Create “macros” triggered when new items are added, for example, when an item is added to a folder, tag it with a keyword, or resync, etc.
I agree that the sooner that plug-in support gets implemented, the sooner that long-requested features will become available from other users. Look what happened with Firefox.