GTFL - A Graphical Terminal For Lisp

Abstract

GTFL is a graphical terminal for Common Lisp. The client is a html page running in a web browser and GTFL provides mechanisms for sending content to the client page from within Lisp (using HUNCHENTOOT and HT-SIMPLE-AJAX).

It is meant for Lisp programmers who want to debug or visualize their own algorithms. Instead of printing tracing information to the Lisp listener (which everybody normally does to understand what's going on), more readable graphical representations of the algorithm's internals are sent to the GTFL client page.

GTFL also comes with mechanisms for visualizing complex (hierarchical) data or control structures. It provides functions for drawing trees and for hiding complexity in elements that expand when the user clicks on them.

Two real-life examples for an application of GTFL can be found here and here. These are debug traces of linguistic parsing and production in the Fluid Construction Grammar (FCG) framework. Such traces help the developers FCG to understand and debug their programs and they help users of FCG to see what their linguistic rules are doing. By encapsulating visualizations for representations in expandable html elements, the complete trace fits into one browser window and still includes every little detail and intermediate processing step of the involved FCG algorithms (which would be thousands of pages debugging output to the listener).

Below is another example of using GTFL for visualizing Lisp data structures. It was created with a few lines of code (examples/asdf-dependency-tree.lisp) and shows the dependencies of an asdf system. Click on the black nodes to reveal details about the systems and click them a second time to hide the details again:

dependencies for asdf system GTFL

description:A Graphical Terminal For Lisp.
long-description:See http://martin-loetzsch.de/gtfl/
author:Martin Loetzsch
licence:BSD-style
version:0.1.3
description:Hunchentoot is a HTTP server based on USOCKET and BORDEAUX-THREADS. It supports HTTP 1.1, serves static files, has a simple framework for user-defined handlers and can be extended through subclassing.
version:1.2.2
description:Base64 encoding and decoding with URI support.
author:Kevin M. Rosenberg based on initial code by Juri Pakaste
maintainer:Kevin M. Rosenberg <kmr@debian.org>
licence:BSD-style
version:3.1
version:0.6.4
no details available
version:0.01
version:2.0.3
trivial-gray-streams
no details available
description:The Common Foreign Function Interface
author:James Bielman <jamesjb@jamesjb.com>
maintainer:Luis Oliveira <loliveira@common-lisp.net>
licence:MIT
version:0.10.6
description:Alexandria is a collection of portable public domain utilities.
long-description:Alexandria is a project and a library. As a project Alexandria's goal is to reduce duplication of effort and improve portability of Common Lisp code according to its own idiosyncratic and rather conservative aesthetic. What this actually means is open to debate, but each project member has a veto on all project activities, so a degree of conservativism is inevitable. As a library Alexandria is one of the means by which the project strives for its goals. Alexandria is a collection of portable public domain utilities that meet the following constraints: * Utilities, not extensions: Alexandria will not contain conceptual extensions to Common Lisp, instead limiting itself to tools and utilities that fit well within the framework of standard ANSI Common Lisp. Test-frameworks, system definitions, logging facilities, serialization layers, etc. are all outside the scope of Alexandria as a library, though well within the scope of Alexandria as a project. * Conservative: Alexandria limits itself to what project members consider conservative utilities. Alexandria does not and will not include anaphoric constructs, loop-like binding macros, etc. * Portable: Alexandria limits itself to portable parts of Common Lisp. Even apparently conservative and useful functions remain outside the scope of Alexandria if they cannot be implemented portably. Portability is here defined as portable within a conforming implementation: implementation bugs are not considered portability issues. * Team player: Alexandria will not (initially, at least) subsume or provide functionality for which good-quality special-purpose packages exist, like split-sequence. Instead, third party packages such as that may be "blessed".
licence:Public Domain / 0-clause MIT
version:0.0.0
description:Ensures consistent *FEATURES* across multiple CLs.
author:Luis Oliveira <loliveira@common-lisp.net>
licence:MIT
version:0.6
description:Babel, a charset conversion library.
author:Luis Oliveira <loliveira@common-lisp.net>
licence:MIT
version:0.3.0
trivial-features
alexandria
trivial-gray-streams
flexi-streams
sb-posix
author:Greg Pfeil <greg@technomadic.org>
licence:MIT
version:0.8.1
alexandria
description:Portable finalizers, weak hash-tables and weak pointers.
author:Luis Oliveira <loliveira@common-lisp.net>
licence:Public Domain
version:0.19
description:MD5 Message Digest function
long-description:This package contains functions to compute the MD5 sum on a stream or string.
author:Pierre Mai
maintainer:Kevin M. Rosenberg <kmr@debian.org>
licence:Public Domain
version:1.8
description:Implementation of RFC 2388
long-description: Contains an implementation of RFC 2388, which is used to process form data posted with HTTP POST method using enctype "multipart/form-data".
author:<jonis@latnet.lv>
version:1.5
description:trivial-backtrace
author:Gary Warren King <gwking@metabang.com>
maintainer:Gary Warren King <gwking@metabang.com>
licence:MIT Style license
version:1.0.2
description:Universal socket library for Common Lisp
author:Erik Enge & Erik Huelsmann
maintainer:Chun Tian (binghe)
licence:MIT
version:0.6.0
sb-grovel
bordeaux-threads
version:0.11.1
description:simple AJAX for Hunchentoot
long-description:See http://martin-loetzsch.de/ht-simple-ajax/
author:Martin Loetzsch
licence:BSD-style
version:0.4
hunchentoot

GTFL comes with a BSD-style license so you can basically do with it whatever you want.

Download shortcut: http://martin-loetzsch.de/gtfl/gtfl.tar.gz.


Sponsored links


Contents

  1. Download and installation
  2. Browser compatibility
  3. The GTFL terminal
    1. start-gtfl
    2. *gtfl-address*
    3. *gtfl-port*
    4. gtfl-out
    5. replace-element-content
    6. append-to-element
    7. reset-gtfl
    8. *reset-functions*
    9. who
    10. who2s
    11. who-lambda
    12. define-css
    13. define-js
    14. make-id-string
  4. Expandable elements
    1. make-expandable/collapsable-element
    2. make-expand/collapse-link
    3. make-expand/collapse-all-link
    4. *create-static-expandable/collapsable-elements*
  5. Tree drawing
    1. draw-node-with-children
  6. Resizing s-expressions
    1. html-pprint
  7. Acknowledgements

Download and installation

GTFL together with examples and this documentation can be downloaded from http://martin-loetzsch.de/gtfl/gtfl.tar.gz. The current version is 0.1.3.

GTFL directly relies on CL-WHO for html generation, HUNCHENTOOT (version >= 1.1.0) for running the web server and HT-SIMPLE-AJAX for the asynchrounous client/server communication. And these libraries themselves require quite a number of other libraries (see the dependency graph above). Make sure you have recent versions of everything.

If you don't want to download all these libraries manually, you can use Quicklisp or ASDF-INSTALL:

(ql:quickload "gtfl")
(asdf-install:install 'gtfl)

Once everything is installed, GTFL is compiled and loaded with:

(asdf:operate 'asdf:load-op :gtfl)

Browser compatibility

As of 2012, all contemporary web browsers except Internet Explorer work well with gtfl.

The output of GTFL is XHTML 1.0 Strict and CSS level 2.1 conform as can be checked below (this page is a document created with GTFL):
   Valid XHTML 1.0 Strict   Valid CSS level 2.1

The GTFL terminal

GTFL consists of two main components that are defined in gtfl.lisp: a html client page running in a web browser and a Lisp web server that mediates between your program and the client page. In order to push stuff from Lisp to the client, the Lisp side of GTFL maintains a "request list" into which the output routines put their content. The client page has a continuously running event loop that polls this list every 200ms using asynchronous AJAX calls.

Here's now some basic usage examples:

(start-gtfl)

This starts the web server. You should see now the client page at http://localhost:8000 (or the address and port you have set).

Now you can push any content there, for example

(gtfl-out (:h1 "hello world"))
=>This will show up in the client page:

hello world

GTFL uses CL-WHO for rendering s-expressions into XHTML. If you are not familiar with CL-WHO then read its documentation first.

More examples:

(gtfl-out (:p "some text, " (:span :style "color:red;" "and some in red."))
          (:p "and a second paragraph"))
=>

some text, and some in red.

and a second paragraph

(defparameter *element-id* nil)

(gtfl-out (:p "a paragraph, " 
              (:span :id (setf *element-id* (make-id-string)) :style "border:1px solid red"
                     "and a span as child element")))
=>

a paragraph, and a span as child element

(replace-element-content *element-id* "and " (:b "new") " span content")
=>

a paragraph, and new span content

(gtfl-out (:p "a paragraph, " 
              (:span :id (setf *element-id* (make-id-string)) :style "border:1px solid red"
                     "and a span as child element")))
=>

a paragraph, and a span as child element

(append-to-element *element-id* ", and " (:b "more") " content")
=>

a paragraph, and a span as child element, and more content

[Special variable]
*gtfl-address*

The address to use for the web server. Default: "localhost".

[Special variable]
*gtfl-port*

The port to use for the web server. Default: 8000.

[Function]
start-gtfl => no values

Starts the web server at the specified address.

[Macro]
gtfl-out &rest expressions => no values

Adds some content to the bottom of the client page.

expressions is something that's ok within CL-WHO's with-html-output macro.

See examples above.

[Macro]
replace-element-content id &rest expressions => no values

Replaces the content of the element with id (a string) by expressions.

See examples above.

[Macro]
append-to-element id &rest expressions => no values

Appends expressions to the element with id.

See examples above.

[Function]
reset-gtfl => no values

Clears the content area of the client page and resets things on the Lisp side.

This function is called either

[Special variable]
*reset-functions*

A list of functions that are called by reset-gtfl. If you want to reset some of your stuff in this case, add your function here: (pushnew #'my-reset-function *reset-functions*).

[Macro]
who &rest expressions => no values

Writes rendered html to *standard-output*. This is a shortcut for (with-html-output (*standard-output*) <expressions>).

expressions is something that's ok within CL-WHO's with-html-output macro.

Example:

GTFL> (who (:div :foo "bar" "baz"))
<div foo="bar">baz</div>
NIL

[Macro]
who2s &rest expressions => html string

Renders expression into a html string. This is a shortcut for (with-html-output-to-string (*standard-output*) <expressions>).

Example:

GTFL> (who2s (:div :foo "bar" "baz"))
"<div foo="bar">baz</div>"

[Macro]
who-lambda &rest expressions => anonymous function

Makes an anonymous function that writes the rendered html to *standard-output*.

Example:

GTFL> (who-lambda) (:div :foo "bar" "baz"))
#<Anonymous Function #x300043366A4F>
GTFL> (funcall *)
<div foo="bar">baz</div>
NIL

[Function]
define-css id css => no values

Adds css fragments to the client page. When the client page is created, all these code fragments are concatenated into one big inline css. Note that you will have to reload the client page when you change css definitions.

id is an id for the code fragment. Repeated calls with the same id overwrite previous definitions. css is the css code fragment. You are responsible for adding line endings.

Example:

(define-css 'foo "
div.foo {font-weight:bold;}
")

[Function]
define-js id js => no values

The same as define-css above, but for javascript code.

(define-js 'bar "
function bar () { return (1 + 1); }
")

[Function]
make-id-string &optional base => string

Creates an uniquely numbered id string that can be used as an id for html elements.

base is the prefix of the id (default: "id").

Example:

GTFL> (make-id-string)
"id-1"
GTFL> (make-id-string)
"id-2"
GTFL> (make-id-string "foo")
"foo-1"
GTFL> (make-id-string "foo")
"foo-2"

Expandable elements

The file expandable-elements.lisp contains functionality to create html elements that expand when the user clicks on them and that collapse again when they are clicked a second time. Furthermore, several elements can be expanded/ collapsed at once using another button.

In order to save browser resources (memory, time), the expanded and collapsed version of each element are kept in Lisp and are only sent to the browser when needed: the version that is initally sent to the client contains only the collapsed version and when the expand link is clicked, it is replaced with the expanded version stored on the lisp side.

Example:

(gtfl-out 
 (:div 
   (let ((expand/collapse-all-id (make-id-string))
         (*create-static-expandable/collapsable-elements* t))
         (make-expandable/collapsable-element 
          (make-id-string) expand/collapse-all-id
          (who2s (make-expand/collapse-all-link expand/collapse-all-id t nil "expand all"))
          (who2s (make-expand/collapse-all-link expand/collapse-all-id nil nil "collapse all")))
         (loop repeat 3
            for element-id = (make-id-string)
            do (htm (:div :style "border:1px solid #aaa;display:inline-block;margin-left:10px;"
                          (make-expandable/collapsable-element 
                           element-id expand/collapse-all-id
                           (who2s (:div (make-expand/collapse-link element-id t nil "expand")
                                        (:br) "collapsed"))
                           (who2s (:div (make-expand/collapse-link element-id nil nil "collapse") 
                                        (:br) (:div :style "font-size:150%" "expanded"))))))))))
=>
expand all
collapse all
expand
collapsed
collapse
expanded
expand
collapsed
collapse
expanded
expand
collapsed
collapse
expanded

[Function]
make-expandable/collapsable-element element-id expand/collapse-all-id collapsed-element expanded-element &key expand-initially => no values

Creates an element that allows to switch between an expanded and a collapsed version.

element-id (a string) is the id given to the element so that function make-expand/collapse-link can reference it. expand/collapse-all-id (a string) is the id of a group of elements that can be expanded/collapsed at the same time (see make-expand/collapse-all-link).

collapsed-element and expanded-element are the collapsed and expanded version of the element. They can be either an expression that evaluates to a html string, e.g. "<div foo/>" or (who2s :div "foo"), or an anonymous function that writes an html string, e.g. (who-lambda (:div "foo")) or #'(lambda () (princ "<div foo/>")). In the latter case the expanded version only gets computed when requested by the client, which avoids unneccessary computation.

When expand-initially is t then the expanded version is shown initially.

Makes a link for expanding/collapsing an element.

element-id is the id of the element to expand or collapse. When expand? is t, then the element gets expanded, otherwise collapsed. title is the title of the link (shown when the mouse is over the link). When nil, then "expand" or "collapse" are used. body is the content of the link (a cl-who expression).

Example:

GTFL> (make-expand/collapse-link "foo" t nil "expand")
<a href="javascript:expand('foo');" title="expand">expand</a>
NIL
GTFL> (make-expand/collapse-link "foo" nil "click here to collapse" (:b "collapse"))
<a href="javascript:collapse('foo');" title="click here to collapse"><b>collapse</b></a>
NIL

Makes a link for expanding/collapsing a group of elements.

expand/collapse-all-idis the id of the element group to expand/collapse. All other parameters as above.

[Special variable]
*create-static-expandable/collapsable-elements*

When this is set to t, both the expanded and collapsed version will be embedded in the html code (one visible and the other hidden). This makes the html code bigger (and thus rendering slower), but allows to save generated html pages, with the expand/collapse functionality still working when not connected to the web server. Note that in the example above this was also set to t, because otherwise the example would not work in this static html page that is not connected to the lisp server.

Tree drawing

In tree-drawing.lisp there is a function for recursively drawing trees. It takes html expressions for drawing a node and its children (which can be trees themselves) and connects them with horizontal and vertical lines. The decision how to layout the tree (i.e. how wide to draw each node) is left to the html rendering engine of the web browser.

Examples (resize the browser window to see the dynamic layout in action):

(defparameter *example-tree* 
  '("top node."
    ("child node one with three children" 
     ("first out of three children") ("second out of three children") ("third out of three children"))
    ("child node two with one child" 
     ("very long text. very long text. very long text. very long text. 
       very long text. very long text. very long text."))))

(defun draw-node (string) 
  (who 
   (:div :style "padding:4px;border:1px solid #888;margin-top:4px;margin-bottom:4px;background-color:#eee;"
         (princ string))))


(defun draw-tree* (tree) 
  (draw-node-with-children 
   (who-lambda (draw-node (car tree)))
   (mapcar #'(lambda (x) (who-lambda (draw-tree* x))) (cdr tree))))

(gtfl-out (draw-tree* *example-tree*))
=>
top node.
child node one with three children
first out of three children
second out of three children
third out of three children
child node two with one child
very long text. very long text. very long text. very long text. very long text. very long text. very long text.
With parameters :right-to-left t :color "green" :style "dotted" =>
first out of three children
second out of three children
third out of three children
child node one with three children
very long text. very long text. very long text. very long text. very long text. very long text. very long text.
child node two with one child
top node.
With parameters :color "#33a" :line-width "2px" =>
top node.
child node one with three children
first out of three children
second out of three children
third out of three children
child node two with one child
very long text. very long text. very long text. very long text. very long text. very long text. very long text.

[Function]
draw-node-with-children node children &key right-to-left color width line-width style => no values

A function for recursively drawing trees in html. It draws a node with connections to it's children (which can be trees themselves).

node is a parameterless function that creates the html code for the parent node. children is a list of functions for drawing the children of the node. The reason to use closures instead of for example strings here is both efficiency (no intermediate structures are built) and to allow for re-ordering of elements (when the tree is drawn from right to left.

When right-to-left is t, then the top node of the tree is drawn at the right. color sets the color of the connecting lines (e.g. "#ff0000" or "red", default is "#888"), width the the width of the horizontal connectors (e.g. "25px", default is "10px"), line-width the the thickness of the lines (e.g. "3px", default is "1px") and style the line style ("solid", "dotted" or "dashed", default is "solid").

Resizing s-expressions

File html-pprint.lisp provides the function html-pprint to display s-expressions in html. This could of course be done with (:pre (pprint x)), but this has the disadvantage that the result will have a fixed width. html-pprint produces output that is very similar to the one of pprint but that also dynamically resizes depending on the available width (while maintaining proper indentation).

Examples (resize browser window to see the dynamic relayout and click on the symbols to highlight other symbols with the same name):

(defparameter *example-list*
  `((("foo-1" "bar-1") (foo-2 bar-2) (:foo-3 :bar-3) 
     (,(make-symbol "FOO-4") ,(make-symbol "BAR-4"))
     (foo-5 . bar-5))
    ((0 1 2 3 4 5 6 7 8 9) ,(asdf:find-system :gtfl))))

(gtfl-out (:div :style "border:1px solid #aaa;" (html-pprint *example-list*)))
=>
((("foo-1" "bar-1") (foo-2 bar-2) (:foo-3 :bar-3) (#:foo-4 #:bar-4) (foo-5 . bar-5)) ((0 1 2 3 4 5 6 7 8 9) #<asdf:system "gtfl">))
(gtfl-out
 (:table :style "border-collapse:collapse;"
         (:tr (loop for i from 1 to 3 
                 do (htm (:td :style "border:1px solid #aaa;"
                              (html-pprint *example-list*)))))))
=>
((("foo-1" "bar-1") (foo-2 bar-2) (:foo-3 :bar-3) (#:foo-4 #:bar-4) (foo-5 . bar-5)) ((0 1 2 3 4 5 6 7 8 9) #<asdf:system "gtfl">))
((("foo-1" "bar-1") (foo-2 bar-2) (:foo-3 :bar-3) (#:foo-4 #:bar-4) (foo-5 . bar-5)) ((0 1 2 3 4 5 6 7 8 9) #<asdf:system "gtfl">))
((("foo-1" "bar-1") (foo-2 bar-2) (:foo-3 :bar-3) (#:foo-4 #:bar-4) (foo-5 . bar-5)) ((0 1 2 3 4 5 6 7 8 9) #<asdf:system "gtfl">))
with :max-width 50 =>
((("foo-1" "bar-1") (foo-2 bar-2) (:foo-3 :bar-3) (#:foo-4 #:bar-4) (foo-5 . bar-5)) ((0 1 2 3 4 5 6 7 8 9) #<asdf:system "gtfl">))
with :max-width 100 =>
((("foo-1" "bar-1") (foo-2 bar-2) (:foo-3 :bar-3) (#:foo-4 #:bar-4) (foo-5 . bar-5)) ((0 1 2 3 4 5 6 7 8 9) #<asdf:system "gtfl">))

[Function]
html-pprint thing &key max-width => no values

Displaying an x-expression in the html. The result lookes like coming from pprint (proper indention), but the layout is dynamic depending on the available width.

thing is the thing to display. max-widthlimits the width of the result (in characters). When nil, then the full available width is used

Acknowledgements

GTFL was initially developed as part of the Babel2 framework (http://emergent-languages.org/).

Joris Bleys and Pieter Wellens have helped a lot in improving the system by finding bugs and suggesting many of its current features.

This page was created with GTFL itself (see examples/index-html.lisp). The layout and structure is heavily inspired by (or directly copied from) DOCUMENTATION-TEMPLATE.

Last change: 2012/01/25 15:05:34 by Martin Loetzsch


Sponsored links