Saturday, February 02, 2008

What Python gets right

In my last post I wrote:

I find that the language itself actually has an awful lot to recommend it, and that there is a lot that the Lisp world could learn from Python.

A couple of people asked about that so here goes:

I like the uniformity of the type system, and particularly the fact that types are first-class data structures, and that I can extend primitive types. I used this to build an ORM for a web development system that I used to build the first revision of what eventually became Virgin Charter. The ORM was based on statically-typed extensions to the list and dict data types called listof and dictof. listof and dictof are meta-types which can be instantiated to produce statically typed lists and dicts, e.g.:

>>> listof(int)
<type 'list_of<int>'>
>>> l=listof(int)()
>>> l.append(1)
>>> l.append(1.2)
Warning: coerced 1.2 to <type 'int'>
>>> l.append("foo")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "container.py", line 126, in append
value = check_type(value, self.__element_type__)
File "container.py", line 78, in check_type
raise TypeError('%s cannot be coerced to %s' % (value, _type))
TypeError: foo cannot be coerced to <type 'int'>
>>>

I also extended the built-in int type to make range-bounded integers:

>>> l=listof(rng(1,10))()
>>> l.append(5)
Warning: coerced 5 to <type int between 1 and 10>
>>> l.append(20)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "container.py", line 126, in append
value = check_type(value, self.__element_type__)
File "container.py", line 78, in check_type
raise TypeError('%s cannot be coerced to %s' % (value, _type))
TypeError: 20 cannot be coerced to <type int between 1 and 10>

I've also got a range-bounded float type, and a length-limited string type.

All of these types have automatic translations into SQL, so to make them persistent you don't have to do any work at all. They automatically generate the right SQL tables and insert/update commands to store and retrieve themselves from a SQL database.

You can't do this in Common Lisp because built-in types are not part of the CLOS hierarchy (or, to be more specific, built-in types do not have standard-object as their meta-type).

You can actually do a surprising range of macro-like things within Python syntax. For example, my ORM has a defstruct function for defining structure types with statically-typed slots. It's syntax uses Python's keyword syntax to define slots, e.g.:

defstruct('mystruct', x=1, y=2.3, z=float)

This defines a structure type with three slots. X is constrained to be an integer with default value of 1. Y is constrained to be a float with default value of 2.3. Z is constrained to be a float, but because it has no default value it can also be None.

If you look as SQLAlchemy and Elixir they take this sort of technique to some really scary extremes.

I like Python's slice notation and negative indexing. Being able to write x[1:-1] is a lot nicer than (subseq x 1 (- (length x) 1)).

I like the fact that hash tables have a built-in syntax. I also like the fact that they are hash tables is mostly hidden behind an abstraction.

I like list comprehensions and iterators.

A lot of these things can be added to Common Lisp without too much trouble. (I actually have CL libraries for iterators and abstract associative maps.) But some of them can't, at least not easily.

Just for the record, there are a lot of things I don't like about Python, starting with the fact that it isn't Lisp. I think syntactically-significant whitespace is a *terrible* idea. (I use a programming style that inserts PASS statements wherever they are needed so that emacs auto-indents everything correctly.) I don't like the fact that importing from modules imports values rather than bindings. And I don't like the fact that I can't optionally declare types in order to make my code run faster.

3 comments:

Bob said...

You didn't mention my two favorite features.

Namespaces. There isn't one global namespace, instead there are lots of modules, classes, and functions, each with its own space. Segregating names into spaces is important in large systems, and Python makes it trivial. This wasn't well-understood when Lisp evolved.

Doc strings in the code. It's like peanut butter in chocolate. (-:

Don Geddis said...

Bob wrote:

There isn't one global namespace, instead there are lots of modules, classes, and functions, each with its own space.

You need to be careful to distinguish Scheme from Common Lisp. Common Lisp doesn't have modules, but it does have symbols in packages.

Segregating names into spaces is important in large systems, and Python makes it trivial. This wasn't well-understood when Lisp evolved.

Common Lisp has been used to build many very large co-resident systems. Putting symbols in packages is a generally effective way in Common Lisp to deal with the issue you've raised.

I don't want to say that packages are the same as modules, but you're greatly overselling the point when you suggest "this wasn't well-understood when Lisp evolved". The designers of Common Lisp had plenty of experience with running huge systems of independent code in the same lisp image.

Doc strings in the code.

Again, Common Lisp already has doc strings in code.

Please note the point of Ron's post. Not "things I like about Python", but instead "things Lisp could learn from Python". You have to actually know what both Python and (Common) Lisp offers in order to suggest reasonable topics.

brian.spilsbury said...

It sounds like you're confusing type and class.
In CL, types are sets of potential objects -- (or integer string).
Classes are objects that describe implementation.

I'd argue that stock CL's support of genericity is quite weak.
The critical issue, though, is not that you can't extend the integer class, but that you can't extend the behaviour of + and so on.
That's a real killer, and it leads to the "well, I'll just reimplement most of CL in a wrapper so that I can make it generic."
On the other hand, python doesn't update objects when you redefine classes.

I'd also argue that python doesn't have namespaces. What python has are modules.
CL has namespaces, but it doesn't have modules.
You can see the difference in something trivial like the 'random' module colliding with a function named 'random' colliding with a variable named 'random' ...

I'd agree that CL needs module support fairly badly.
It would also be nice if python had namespaces. :)

(hopefully this isn't posted twice)