User Interface 101: UI Data Binding

The traditional method of writing UI logic is hard. It requires writing an enourmous amount of housekeeping code, such as event handlers and update methods. Non-interactive testing is almost impossible to realize.

I propose a method to bind an user interface to data using only a set of declarations, describing data constraints and UI/data relations. This set is called "binding".

Here are a few applications for bindings:

• A property view, allowing to select items from a list and change their attributes.
• A spatial view, allowing to position and resize items on a surface.
• A complete application made of cascading spatial and property views.
  

A binding connects a model and a graphical UI in both directions. It can be described in an XML document using a dedicated DTD scheme, obsoleting imperative micromanagement of dialogs.

Both model and UI should be reflective: their properties and elements can be enumerated and queried by the binding. If either model or UI is not reflective, additional XML documents are required, providing information about structure and constraints.

The model can be of any of these following format types:

• SQL database
  
• XML DOM
• JSON tree
  
• Python class tree
  
• C structures
  

The binding will initially support following UI frameworks:

• GTK+
• Glade
• Some canvas (Clutter, GooCanvas?)

A binding supports following model datatypes:

• Numbers (integers, floats)
• Strings
  
• Lists of mixed types
  
• Compounds (structures, classes)
  
• References
• Functions
  

On the UI side, a binding supports following display and editing methods:

• Text/numeric entries
• Combo/choice boxes
• Radio buttons
• Check boxes
  
• Buttons
• List views
• Tree views
• Sliders
• Scroll bars
• Progress bars
  
• Canvas items

More will follow.

User Interface 101: Device Independent User Input Processing

This is the first post of a new series called "User Interface 101", where I try to invent a functional, description-based approach to writing graphical user interfaces bottom-up, devoid of any imperative baggage. The first post deals with one of the basic building blocks of human-computer interaction, user input.

Users operate computers only through input devices. An input device, no matter what shape, can be divided into separate elements, of which each element carries a value and an activity state.

Internally, we are going to represent each input device element as an abstract "input source", which is designed to yield any information a computer program would require to successfully interpret user input, while burdening programmers with device specific interface details only if necessary.

An input source describes a single input device element such as a key on a computer/MIDI keyboard, a button on a mouse or joypad, an IR pointing device, a joystick or a finger on a touch pad. For multi-dimensional input devices (e.g. mouse, pen, touch pad), each input source describes a single dimension.

An input source yields one single (signed integer) value at a time contained within a hardware dependent range r0..r1, where r1 > r0. This enables tracking of both digital key/button strokes/presses/clicks and analog stick/mouse/pen/finger movements for a single dimension.

An input source may also yield an activity state, if supported by the device, which indicates if the input device element is enabled/alive/in use. This enables tracking of caps/scroll/num lock key states, cursor dead zones and retraction of pen/finger/device.

An input source may optionally point to a descriptor containing information that may help to identify the source and interpret the values.

An input source may be tagged as relative or absolute in its descriptor. Relative axes yield delta values. Adding delta values subsequently, one may acquire an absolute value, contained within an unspecified range. This enables tracking of rotary wheels and mouse movements while repositioning the device.

An input source may describe a glyph or command it represents in its descriptor. This enables querying a scan code or button index.

An input source may describe an one to three dimensional position vector in its descriptor, indicating the sources spatial position on the device in meters, relative to either the center or a corner of the device. This enables to map tasks to input sources based on their adjacency and location.

An input source may describe the dimension it operates on in its descriptor, if it is part of a composite source. Zero indicates no specific dimension. This aids with building multi-dimensional values from multiple input sources.

An input source may describe a default resting point or range in its descriptor that the control returns to when not operated by the user. This enables to determine whether a key/mod wheel/button/axis is resting.

Imperative GUI Programming Sucks

I have recently had my eyes on Haskell, and I liked what I saw. I spent about 20 years of my life writing programs in imperative tongue, and the more I knew, the more trivial the solving of problems became, the more I found the language to be obstrusive and low level.

Why do I need to be bothered with rolling out the exact details of a routine, when I have exact logic relationships in my head? Can't I put my understanding of the abstracts in letters and the compiler figures out all the implications and details?

Turns out, functional programming, even if it looks a bit ugly at times, does exactly what I want. Haskell is a functional programming language, where the specification is the program. With such a concept, you theoretically would write even less code than you would in Python, while making even less mistakes.

But this world is not quite ready for functional approaches. All that old bullshit comes back through Haskells imperative back door of monads and I/O operations, and if you do a lot of interfacing, there is right now virtually no difference between Haskell and Python or C++. I had a look at the GTK+ binding for Haskell and it outright sucks. There I am again, writing event handlers and filling GUI's sequentially. The specification is again not in the program, but in my head.

As a GUI programmer, how can I get closer to writing a GUI cleanly from specification, without muddling much with the details?

Since the web is said to be the future, I had a look at different web technologies. There is a functional language in use on the web for quite a time, and it is called XSLT. This XML based descriptive language specifies how an XML can be translated into something else, either another XML or a completely different format. No imperative I/O. Just pattern matching and templates, and the XSL transform does the rest. You can even transform the XSL document itself using XSLT.

You could build a GUI from this. One part of the job, at least, in a primitive way. You could turn the XML into an HTML form using pure XSLT, but there is still the problem of turning user input back into XML. You would also want to update only parts of the source DOM on user input, along with various validation mechanisms. Source data and front end would have to be connected in both ways based on a formal specification. I searched the web up and down for this "two-way XSLT", but found nothing yet.

Is it really possible that in the year 2009 there is no need for a functional approach to binding data and front end? Or is it simply too hard, or silly to think of? I can't believe that. From my perspective, all these different approaches to editing data - buttons, edit fields, lists, trees, knobs, sliders, graphs - boil down to about at most 30 primitive concepts in combination, reimplemented over and over, requesting each and every GUI programmer ever coming along to stupidly assemble every dialog as if it was the first he had ever written. The technology we use today does not scale. It should become easier the more interfaces you do, instead it becomes tedious. Refactoring is a nightmare because GUI tests are way too hard to write.

If I only were able to write down the specifications of an GUI and get my working program right away, testing steps would be self-evident, they could even be automatically determined. Changing a GUI flow would become a lot easier. I would literally simply change connections instead of rewriting dialogs and tests. Somebody told me: everything that is hard to do is worth doing. Screw this guy. This stuff should be trivial. Figuring out the specification should be the hard work, not working like a code monkey.

Imagine how easier experimentation with new paradigms would become if we had enough time to test different ideas for user interfaces. How utterly convenient and elegant these applications would be, regarding both code and usage.

If you know of such a project in the works, please let me know. I am desperate.