Planet Scheme

Syndicate content
Updated: 13 weeks 6 days ago

Doug Williams: Homogeneous, N-Dimensional Arrays (ndarrays)

Thu, 03/27/2008 - 11:57
The basic representational element for PLT Scheme Schemelab is a homogeneous, n-dimensional array - or ndarray. This post will discuss the initial design for ndarrays and discuss some of the operations on them.

Internally, an ndarray is represented by a structure with the following elements:
  • ordering - (symbols 'row 'column), specifies the order in which dimensions are stored ('row for row major and 'column for column major). Generally, this isn't that useful except for (eventually) interfacing with foreign code (C uses row major and FORTRAN uses column major). Also, the transpose operation converts from one to the other (although that is more a side effect that its real functionality). [read only]
  • shape - (listof natural-number/c), specifies the shape of the array. The length of the shape list is the number of dimensions for the array and each element is the cardinality of the corresponding dimension. This may be set to reshape the array.
  • ndim - natural-number/c, the number of dimensions for the array. This is equal to (length shape). [read only]
  • size - natural-number/c, the total number of elements in the array. [read only]
  • disp - natural-number/c, the displacement (in elements) of this subarray is the data vector. This will be zero for any ndarray that owns its own data. [read only]
  • strides - (listof natural-number/c), a list of the number of elements to skip to move to the next element in the corresponding dimension. [read only]
  • offset - natural-number/c, the offset (in elements) for each addressed element. This will be zero for any ndarray that owns its own data. [read only]
  • data - typed-vector?, the typed vector containing the data for the array. [A typed vector encapsulates an SRFI 4 vector (for u8, u16, u32, u64, s8, s16, s32, s64, f32, or f64 arrays), a complex vector (for c64 or c128 arrays), or a Scheme vector (for Scheme object arrays).] [read only]
  • base - (or/c array? false/c), the base array for this (sub)array or #f if the array owns its own data (i.e. it is a base array). Note that the referenced array may itself base a non-#f base entry. [read only]
Note that I may not need both the displacement (disp) and offset fields. But, the general idea for array iteration is to add the displacement once when the iterator is initialized and to add the offset for each element. It might be possible to collapse those into one offset without loss of generality.

The most general array constructor is make-array, which has the following contract:


(->* ((listof natural-number/c))
(#:vtype (or/c vtype? symbol?)
#:ordering (symbols 'row 'column)
#:fill any/c)
array?)


The required argument is the shape of the array. The optional keyword arguments are #:vtype, which specifies the type of the array (default object), #:ordering, which defaults to 'row, and #:fill. If #fill is not specified, the array elements are not initialized.

Example:

(define a1 (make-array '(3 2) #:vtype f32 #:fill 0.0))

Creates an array whose shape is '(3 2) (i.e. three rows and two columns) and whose elements are 32-bit floating-point numbers that are initially 0.0.

A fully qualified reference for the array returns the corresponding (scalar) element. For example, (array-ref a1 '(1 1) returns the value of the element at '(1 1).

A partially qualified reference for the array returns a reference to the corresponding subarray. For example, (array-ref a1 '(: 1) ) returns the subarray of shape '(3) that corresponds to the second column of a1. [Note that this is a reference to the second column of a1 and not a copy of it. Array referencing never copies anything.] Likewise, (array-ref a1 '(1 :)) [or equivalently, (array-ref a1 '(1))] returns a reference to the second row of a1.

Array mutation also allows partially qualified references. In that case, the value specified must be broadcastable (described in a future post) to the shape of the reference. For example, (array-set! a1 '(: 1) 1.0) sets the elements of the second column of a1 to 1,0 (yes, the scalar 1.0 is broadcastable to shape '(3)). Likewise, (array-set! a1 '(1) '(4.0 8.0)) sets the elements of the second row of a1 to 4.0 and 8.0.

A complete set of array manipulation functions will be provided. This will include: build-array, array-map, array-map!, array-for-each (i.e., similar to what SRFI 43 provides for vectors).

I haven't though about the semantics for things like array-fold and array-unfold. They may be well-defined someplace - I really haven't looked. If anyone knows where or has any ideas, please let me know.

Note that it is not my intention to define functions like add, subtract, multiply, divide, ... (as numpy does for Python). Rather, we will rely on the array mapping functions to extend scalar operations to arrays.

Doug Williams: Schemelab Concept

Thu, 03/27/2008 - 10:08
My PLT Scheme project for this year is to create a Schemelab collection to provide better analysis capabilities in PLT Scheme. It will be something of a mini-Matlab with capabilities similar to what numpy, scipy, and matplotlib provide in Python. I plan on making a new numeric collection to provide homogeneous, n-dimensional arrays (and, as a subset, matrices) as the primary underlying representation. Next, the science collection will be updated to use the new numeric package. Finally, a new plot collection will be developed to provide better visualization functionality. [After these are done I'll also update the simulation and inference collections to use the new Schemelab packages.]

I've already started work on the numeric collection. The basic representational element is a homogeneous, n-dimensional array (ndarray). An ndarray's type and shape are specified when it is created. The type may be any of the element types allowed for SRFI 4 vector types (u8, u16, u32, u64, s8, s16, s32, s64, f32, or f64), a complex type (c64 or c128), or may hold any Scheme object (object). The shape is a list of natural numbers where the length of the shape is the number of dimensions for the ndarray and each element is the cardinality of the corresponding dimension. References to array elements will support array slicing (in any dimension) for both array accessing (array-ref) and mutation (array-set!). Slicing operations create new views of the referenced array as opposed to copying portions of the array. I will start posting entries on different aspects of the numeric collection in the next few days.

The updates to the science collection to support (or to make use of internally) the new numeric collection should be relatively straightforward. The main issue will be to retain compatibility with the existing data types (i.e. vectors). Most of the really numerically complex code (e.g. special functions and random number distributions) will not be affected since they don't generally provide vector or array operations. The main changes will be to the statistics and histogram modules, which will be modified to work with ndarrays as well as vectors. Finally, some of the modules (e.g. histograms and ordinary differential equations) will benefit from being reimplemented using ndarrays internally.

I've also prototyped some code for the new plot package that provides much more functionality than the current PLoT package. The initial capability will be very much patterned after the functionality provided by Matlab (or more precisely, matplotlib for Python, which is also based on Matlab). It provides precise control over the elements of a graph (or multiple subgraphs). It also provides interactive graphics functionality for more dynamic analysis capabilities.

Note that all of these new (or updated) collections require PLT Scheme Version 4 (currently 3.99) and are not compatible with earlier versions. As such, I am not releasing them to PLaneT until either PLT Scheme Version 4 is officially released or there is a separate PLaneT repository for PLT Scheme Version 4 code. I will be putting the code on the Schematics web site at some point.

Arto Bendiken: Finding Clojure, or How I Learned to Stop Locking and Eval the JVM

Wed, 03/26/2008 - 19:00

Ever since the Scheme R6RS roadblock, I've been looking to branch out into other Lisps outside the Scheme ecosystem. The good news is that there's no shortage of nascent Lisps and Lisp-wannabes, with many interesting new undertakings being launched month upon month.

There's one in particular, called Clojure (pronounced closure), that was unveiled recently and effortlessly stands out from the bunch. There's much to like about Clojure, but first and foremost, it follows a principle I'm very fond of: code talks, bullshit walks.

That is to say, it's not difficult to imagine any number of interesting directions to take Lisp into, nor is it generally all that much effort to implement proof-of-concepts of such ideas (it's Lisp, after all); but to actually create a full integrated, practical and performant system, that's another matter entirely.

The difference between toy and tool is more than quantitative, and the leap from an SICP metacircular interpreter to something deserving of the hallowed title "Lisp system" is non-trivial, requiring not just deep mastery of the fundamentals but also the discipline and opportunity to take care of all the "boring" stuff so as to ship it. As I know all too well from plenty of practice, this last so-called 20% is where most efforts ultimately wither out to oblivion.

Thus it's a pleasant surprise to discover that while Clojure has no shortage of goodies, it's also remarkably mature and usable right here and now. The author, Rich Hickey, says that he has worked on it for two years, and I think it's admirable that he kept tight-lipped about it for the duration; as a first alpha release, Clojure as it is currently available simply rocks. No Marimba phenomenon, here.

As for the goodies, how does a concurrent, functional Lisp1 with software transactional memory sound like? To me, it sounds like the future. (And that's just the tagline, there's much more.)

I've thought for a while now that if you took a purely functional subset of Scheme (pre-R6RS), sprinkled it with the best of Haskell (without the whole mandatory bondage and discipline, thanks) and built it all on top of an Erlang-like runtime, that might form a pretty compelling basis for a serious attempt at a new practical Lisp dialect. Clojure is perhaps not quite the particular mix that I had in mind, but it's very much in the ballpark.

Indeed Clojure embodies the kind of Lisp I'm interested in, that is, a pragmatic go-getter system actively drawing on the best new computer science research and pushing Lisp ever further out on the frontier of the future, instead of complacently sitting around arguing about how many lambdas can dance on a parenthesis while wondering why the competition is drawing nearer.

Considering how new Clojure is, there is a commendable amount of documentation already available, and the user community is growing in leaps and bounds, drawing both Common Lispers and Schemers as well as a good number of people entirely new to Lisp. The momentum that Clojure has going is tangible.

On the flip side, and these are really pretty serious omissions but may perhaps be intractable on the JVM at present, Clojure lacks tail recursion optimization and continuations. Also, while the concurrency features are impressive, I'm as of yet unsure how well-suited Clojure's JVM-based threading might be to a massively parallel programming style such as what I've been using with the lightweight processes in Gambit and Erlang, both of which will run millions processes without flinching. The author explains that these use cases are kind of supported but perhaps not really Clojure's sweet spot.

Now, given these JVM-imposed constraints, I should admit that I put off looking into Clojure for several months just given that it's based on the JVM; not only have I not recently had any reason to use the JVM, but my antipathy to Java is long-standing and well-documented.

However, in his LispNYC talk, Rich Hickey makes compelling points about why he based Clojure on the JVM, and I can't help but agree with many of them. Similarly, Neal Ford in his superb recent JRuby-related interview on JavaWorld makes a case that the actual positive legacy of Java will be the JVM as an established and widespread high-performance platform and infrastructure for enabling polyglot programming with the already more than 200 languages (including JRuby, Rhino, Jaskell and Jython) that you can run on it. I can buy that argument, too.

So, I'm coming to the unwelcome conclusion that my distaste for Java the language may have blinded me to the potential of the JVM as a platform for hosting other more powerful programming languages. I haven't much followed the JVM's evolution for years now, but intend to keep a closer eye on it in the future - especially the ongoing work around JSR 292 that aims to make the JVM suck less for hosting dynamic languages.

In any case, I'll be taking Clojure out for a spin. It's definitely a singularly unique new Lisp well worth looking at and learning from. I highly recommend Rich Hickey's two-hour LispNYC presentation which I can only describe as delightfully competent. (Also, in case STM is an unfamiliar concept, Simon Peyton-Jones's brief introduction at OSCON last year should be able to fill in the gaps some.)

Untyping: Epic Lolz

Tue, 03/25/2008 - 12:25

Over Easter we insulated our roof. It was quite easy job, but I'm not going to write much about it as this post is entirely an excuse to show this picture:

funny pictures

No animal abuse was involved: he climbed up the ladder into the loft and stayed there until we moved him to put down more insulation.

Dave Herman: Makefile for HUnit tests

Mon, 03/24/2008 - 01:09
Here's a nice idiom for running HUnit tests from a Makefile. Jesse Tov gave me the idea that you can use make's dependencies to test only modules that have changed since the last time you ran the tests. Pick a standard name for each module's tests--tests, say--and make sure it's defined in the first column (since we'll be searching for it with a simple grep). Every time we test a module, we'll touch a dummy file of the same name in some hidden directory--call it .test. So we have two lists of files, the modules of the application and the test dummy files:
SOURCES := $(wildcard *.hs) $(wildcard *.lhs)
TESTS := $(foreach src,$(SOURCES),.test/$(src))
To run the tests, first we collect the list of modified modules and save the names of their tests (Foo.tests, Bar.tests, etc.) in a temporary file .test/args.
.test/%hs: %hs
@if grep '^tests ' $< > /dev/null 2>&1 ; then \
touch $@ ; \
echo $* | sed -e 's/\..*/.tests/' >> .test/args ; \
fi
Now every time we run the tests, we first make a fresh .test/args file, and then we run GHC with a command-line option to evaluate an expression that runs those tests:
test: teststart $(TESTS)
@echo ghc Test -e "\"Test.run [ `xargs -a .test/args | tr ' ' ','` ]\""
@ghc Test -e "Test.run [ `xargs -a .test/args | tr ' ' ','` ]"

teststart:
@mkdir -p .test
@$(RM) -f .test/args
@touch .test/args
It's also useful to have a target that always runs all the tests:
retest: testclean test

testclean:
$(RM) -rf .test
This assumes the existence of a simple test harness module:
module Test (run) where
import Test.HUnit
run :: [Test] -> IO ()
run ts = (runTestTT $ TestList ts) >> (return ())

Dave Herman: Some tools just don't like each other

Sun, 03/23/2008 - 17:01
brit·tle, adj.
  1. Multi-line string literals in Haskell require so-called "string gaps": one '\' character to terminate a line and another '\' to start the next line.
  2. With GHC, Haskell programs may be preprocessed with CPP, which coincidentally strips the "\ ... \" characters from the source, resulting in an illegal Haskell string literal.
  3. Mercifully, it also happens that CPP doesn't strip the characters if the first '\' character is followed by a space before the newline.
  4. But of course, a commonly used feature of emacs is to silently strip trailing whitespace at the end of lines on every save.
  5. Not that you can see the difference, given the well-known human limitations at visually distinguishing whitespace.
(Sensitive types should take this neither as a criticism of GHC nor as implicit condonement of using CPP for Haskell programs--I know it's shunned. You've gotta admit this is beautiful, though.)

Dave Herman: When to use point-free style

Fri, 03/21/2008 - 13:56
I've often struggled with the question of when/whether to use point-free style. It can be dangerously addictive, especially in languages with syntactic support for it like Haskell. But it's notorious for creating dense and impenetrable code. Yesterday I linked to advice from the GHC community on when not to create needless abstractions, advice which could be applied when considering a point-free abstraction (man, does it ever take self-restraint not to go wild with the puns here).

The reason why pointful style can be so helpful is it allows us to think about the definition of a computation in terms of particular examples. For any number, let's call it x, ... This is also why set comprehensions in math, and consequently list comprehensions in programming languages, are so approachable: they describe a set by appeal to representative examples.

I think the key to successfully using point-free style is when you want to treat a function itself as a single data point. For example, if you're sorting a list with some comparison function, you don't want to have to drop down a level of abstraction to think about the individual pairs of elements being compared; you just want to think about the comparison function (like "numeric" or "reverse alphabetical") as the object of your attention.

Dave Herman: Well said

Wed, 03/19/2008 - 18:34
From the GHC wiki:
It's much better to write code that is transparent than to write code that is short.

Often it's better to write out the code longhand than to reuse a generic abstraction (not always, of course). Sometimes it's better to duplicate some similar code than to try to construct an elaborate generalisation with only two instances. Remember: other people have to be able to quickly understand what you've done, and overuse of abstractions just serves to obscure the really tricky stuff.

Untyping: Welsh's First Corollary to Weakliem

Fri, 03/14/2008 - 16:57

Weakliem’s First Rule of Application Development states, roughly, that design is less important than functionality. While I agree in principle I think his proof is lacking in a number of places. Specifically, he states: “Recall that when Google first appeared, most search engines embraced the design philosophy still in evidence at MSN.com: bright and noisy, yet roughly equivalent in functionality. Google was positively audacious in both its austerity and its function. ... Similarly, My employer’s website is frequently ridiculed for being amateurishly designed” What I think he forgets is all design, however amateurish, still conveys something. Google's (to my eyes incredibly ugly) logo said “hey, we're a bunch of geeks having some fun” which exactly matches the company culture and helps attract all those PhDs that Google employs. Similarly Gordon's employer's website looks like it was designed by someone's cousin, but that is the right look for its clients. It gives the website credibility with the consumers who put down a large chunk of change for a holiday they can only afford once a year. Good design is design that is right for the target audience, which can be something very different to aesthetically pleasing design.

Dave Herman: Not fade away

Fri, 03/14/2008 - 13:16
From the old-languages-never-die dept.:
Obsolete programming languages such as FORTRAN and COBOL are still widely used.

-- J.J. Duby, 1971

Dave Herman: Another cute Haskell function

Mon, 03/10/2008 - 17:12
join :: Eq a => [(a,b)] -> [(a,c)] -> [(a,(b,c))]
join a1 a2 = [ (a,(b,c)) | (a,b) - a1, c - [ c | (a',c) - a2, a == a' ] ]

infixr 5 ⋈
(⋈) :: Eq a => [(a,b)] -> [(a,c)] -> [(a,(b,c))]
(⋈) = join

-- for example:
[("a",1),("b",2),("c",3)] ⋈ [("a",'a'),("b",'b'),("c",'c')]

Dave Herman: Crazy debugging feature idea

Fri, 03/07/2008 - 19:14
Here's a crazy idea. Closures are like objects with private fields, right? Specifically, outside of the function body, you're not allowed to refer to the variables in the closed environment. But one of the most annoying things that happen when I'm debugging is that I want to call some local function, but it's not available from the REPL:
(define (foo x y)
(define (helper z)
... x ... y ...)
...)
What if, at the REPL (i.e., in debug mode only), closures were openable? You could use dot-notation to refer to them, and all you'd have to do is somehow provide the extra bindings needed to fill in their environment. Keyword arguments, maybe?
> ((close foo.helper [#:x 42] [#:y 'blah]) "value of z")
'result-value-ftw!

Untyping: Of Interest 07/03/2008

Fri, 03/07/2008 - 17:05
  • TRIZ is a methodology for creativity. Worth looking at.
  • The iPhone SDK has got every geek drooling over his keyboard, and for good reason. I think this model will work.
  • Is 1000 True Fans the path to happiness? The argument goes that 1000 true fans will provide sufficient income for an artist, and that isn't that many people. Assuming your work requires only you to produce it this seems a plausible number, but I think the scarcity of true fans is a big roadblock. You might need 100,000 fans before you get 1000 who are really into what you do.
  • Looking at what you can do with rather more than 1000 true fans, interesting workplace experiments from 37Signals. I like the four-day week; funding passions is too normative; the release of the iPhone SDK means discretionary spending accounts gets the thumbs up from Dave.

Dave Herman: Xor

Wed, 03/05/2008 - 11:39
While I'm at it, let's continue the series of logical Maybe operators with xor:
infixr 2 ^^^

(^^^) :: Maybe a -> Maybe a -> Maybe a
Nothing ^^^ r@(Just _) = r
l@(Just _) ^^^ Nothing = l
_ ^^^ _ = Nothing

Dave Herman: My current favorite little Haskell function

Wed, 03/05/2008 - 11:20
Here's my pet Haskell combinator du jour:
infixr 2 |||

(|||) :: Maybe a -> Maybe a -> Maybe a
Nothing ||| x = x
x ||| _ = x
This gives you a concise notation for branching in the Maybe monad, so instead of having to write:
case x `lookup` env1 of
Just v -> Just $ f1 v
Nothing -> case x `lookup` env2 of
Just v -> Just $ f2 v
Nothing -> Nothing
you can write:
do { v - x `lookup` env1; return $ f1 v } |||
do { v - x `lookup` env2; return $ f2 v }
This gives you the same kind of idiom that Schemers use or for. Come to think of it, I guess you could also write the analog of and:
infixr 3 &&&

(&&&) :: Maybe a -> Maybe a -> Maybe a
Just _ &&& r@(Just _) = r
_ &&& _ = Nothing
I haven't used this one yet, but it seems natural.

Dave Herman: Intentional capture

Tue, 03/04/2008 - 18:34
Procedural hygienic macro systems like the syntax-case system make it possible to write capturing macros--macros which, depending on your philosophy, you might call "non-hygienic." The classic example is the "anaphoric" conditional form if-it, which implicitly binds a variable it to the result of the test expression:
(if-it 42 (+ it 1) #f) ; => 43
The difficulty in getting such a macro right comes when you try to write another macro that expands into if-it. To quote the mzscheme manual's section on macros, "macros that expand into non-hygienic macros rarely work as intended."

Andre van Tonder's SRFI 72 document contains a perfect and concise example, due to Kent Dybvig, of two different ways a macro might expand into a capturing macro. On the one hand, we might want to write when-it, a simple "one-armed" conditional that implicitly binds it in the same way as if-it:
(when-it 42 (+ it 1)) ; => 43
On the other hand, we might want to use if-it to implement the hygienic or macro, which shouldn't capture any variables.
(let ([it 10]) (or #f it)) ; => 10
First, here's the implementation of if-it: we create an identifier for it with the same lexical context as the operator of the expression:
(define-syntax (if-it stx)
(syntax-case stx ()
[(op e1 e2 e3)
(with-syntax ([it (datum->syntax #'op 'it)])
#'(let ([it e1])
(if it e2 e3)))]))
The references that will be captured by the introduced binding of it are the ones that were introduced into the program in the same expansion step as the occurrence of if-it in the macro call. In particular, if the occurrence of if-it was in the original program (i.e., written explicitly by the programmer), it captures references to it that were in the original program; if the occurrence of if-it is the result of a macro expansion, it captures only those references to it that were generated in that same expansion step.

This means that a hygienic macro that expands into if-it will work as expected:
(define-syntax or
(syntax-rules ()
[(op e1 e2)
(if-it e1 it e2)]))
Since the reference to it appears in the same expansion step as the occurrence of if-it, that reference is captured, but no references to it within subexpressions e1 or e2 (which had to have already been there before this expansion step) are captured.

If you want to write another capturing macro that expands into if-it, it's a little more work. Essentially, you have to capture it all over again. The moral of the story is that you always have to ask explicitly for a macro to capture an introduced identifier.
(define-syntax (when-it stx)
(syntax-case stx ()
[(op e1 e2)
(with-syntax ([it* (datum->syntax #'op 'it)])
#'(if-it e1 (let ([it* it]) e2) (void)))]))
Here we once again create an identifier with the same lexical context as the operator, and we bind it to the occurrence of it introduced by if-it.

These are good defaults for a hygienic macro system: it's easier to write hygienic macros but still possible (albeit a little harder) to write macros that capture. This is even true when you abstract over capturing macros: macros that expand into capturing macros are hygienic by default, but with a little more work again, you can create capturing macros that abstract over other capturing macros.

Untyping: Of Interest 03/03/2008

Mon, 03/03/2008 - 09:53
  • This Wired article, though not very deep, gives a nice overview of Internet business models.
  • I don't normally like rants, but found this one well written and amusing. Also, the Jakob Neilsen style bolding of important phrases: I think it works. Look for more strong on Untyping in the future.
  • We're going to London on Wednesday presenting at QMUL on the work we've been doing for the School of Biological and Chemical Sciences. If you fancy meeting up, drop me a line and we'll see if timetables can sync.

Untyping: Requiring up and down syntax levels

Mon, 03/03/2008 - 09:44

If you do any macro programming in PLT Scheme you are sure to run into the dreaded “no #%app syntax transformer is bound” error message at some point. Though puzzling, the fix is actually quite simple in almost all cases. Assuming you're using 3.99, you either need to:

  1. (require (for-syntax scheme/base))
  2. (require (for-template scheme/base))

What the error means is that some syntax has expanded in a function application, but #%app, the PLT Scheme primitive that actually handles application, is not bound in the phase in which the syntax is being evaluated. Requiring for-syntax will bind #%app in the phase before the current evaluation phase, while requiring for-template will bind #%app in the phase after. In most cases you want for-syntax. However, if you are writing functions that return syntax that is then inserted into a program (such a function would be required for-syntax elsewhere) you must use the other form, to make sure the syntax has #%app available to it.

Untyping: Happy Valentine's from Untyped

Mon, 03/03/2008 - 09:44

For Valentine's Day we have created a new website, Smut Shorts. If you have something to say about love or lust, and can do so in 500 characters or less, then please add it to the growing number of “shorts”. It's anonymous and fun. Just, no porn thanks.

If you're reading this site, you're probably interested in the technical details behind Smut Shorts. It is a collaboration between a number of people, most of whom have chosen to be anonymous. The majority of the coding was done by yours truly, and therefore in Scheme. It is running on the PLT Scheme webserver (version 3.99) and uses PostgreSQL as the back-end. I coded it up in about two days. It was a side-project, so it was a bit of rush job and there is lots still to do. If you break the site let me know and I'll try to fix it.

A few interesting lessons were learned from doing this site. It all comes down to scalability, which is something that has recently been on my mind a lot. In this case we want to scale down to the low end — the guy who is just hacking up something in his spare time and wants to get it done in a hurry. Our frameworks, Snooze and Lylux, are pretty good but they don't support a fast start. You have to create a whole bunch of files before you've even got your first page up. Furthermore, we've always avoided creating a templating mechanism, as we've said that we'd rather use smart people who can balance parens than create this unnecessary divide between designers and programmers. I now recognise this is a mistake. Had we a templating mechanism I could have pushed more design work to my collaborators. It's not that they're stupid (far far from it) but they're busy and don't have time to learn even Scheme basics just so they change a few lines of text. If we're gonna grow the Scheme web-hacking community it has to start with dudes messing around in their spare time, so we need to address the low-end of scale. The high end of scale can wait till the IPO ;-)

Untyping: SPIN-Farming, Franchising, and the Future of Software Frameworks

Mon, 03/03/2008 - 09:44

Ever thought about buying a franchise? Like the idea of running your own business but don't want the risk of trying it all alone? How about farming? Attracted by the notion of growing all your own food, and connecting with nature? How about both — franchised farming? Sounds unlikely, but that's essentially what SPIN farming is.

The core of SPIN farming is, well, farming, but not on the scale that most people associate with modern farming techniques. SPIN farmers typically work plots less than an acre in area, and achieve good returns by concentrating on the most profitable crops, and utilising crop rotation to increase yield. The best description I've found is here, though it isn't very detailed.

In itself the farming techniques aren't that radical. I remember learning in high school that crop rotation was one of the key innovations of the agrarian revoluation, and that was some time ago (both high school, and the agrarian revolution). What is novel is the way the SPIN farming business is run, and that's what causes me to call it a franchise. Contained in the guide books (the complete set can all be purchased online) is everything you'd expect from a good franchise: a business plan, marketing advice, and a detailed day-to-day workflow. In standardising the product and creating a reproducible process it really isn't any different from McDonalds. Now SPIN farming isn't a true franchise — you don't buy the right to use the name, and there isn't any ongoing fee. And there's no equivalent to Hamburger University either. At least not yet. It is still an interesting business model and one that I think has great potential, though perhaps not for the financial gain of the founders.

The franchise idea, believe it or not, has great relevance to computing. What is convention over configuration if not the computing equivalent of the franchise's reproducable process? Perhaps by regarding frameworks as franchises we can shift the emphasis from technical development to supporting the developer in every way they need to succeed. This seems to me a better goal for both framework developers and users.