2010 October 25 10:19am
Aaron Bedra - Clojure in the field
A couple of the main issues to address:
- Web frameworks - too many to pick from, need to coordinate these and need better docs
- Dependency management - Lein is a good start, but we need more. Also Clojars is too hard to search.
Stuart Sierra - Macro Club
slides
clojure.test uses macros. This is confusing at times. For example, “thrown?” is not a var. It is interpreted by the “is” macro. This is surprising because it looks like a function call when it is used and this is one of the down-sides of macros: surprising behavior.
lazytest is Stuart’s replacement for clojure.test. He went through macros, protocols, etc. looking for good abstractions for tests and ended up falling back to the powerful, core Clojure constructs:
- functions - represent tests
- sequences - represent test suites
He is still struggling with test contexts that perform setup/teardown around tests. The problem is having a good way to get the context state into the tests. There are macros to accomplish this but they are surprising because they establish new scoping rules.
In summary:
- use Clojure’s core types as abstractions (functions are the most powerful)
- use macros for sugar
- don’t violate user’s expectations
Lightning Talks
Zach from Runa talked about Aleph. An asynchronous web server built on “channels” which are unidirectional pub/sub queues.
Alex Miller from Revelytix described how to use Clojure zippers to process trees of Records. Great talk.
Infer from Bradford Cross is a very approachable machine learning library for Clojure.
limelight is a rich client GUI framework.
Hazelcast is a Java dynamic clustering library.
Mark McGranaghan - Ring
Model web requests and responses with Clojure concepts:
- app( request-map ) -> response-map
- the app is a Clojure function
- the request and response are Clojure maps
- “middleware” is implemented as higher-order-functions
They produced ~120 line “spec” and from there an entire ecosystem of libraries and frameworks has grown. Now it is possible to deploy a Ring app that mixes in components from a variety of locations and they all work together in one stack.
The essence of Clojure is composable abstractions
If you want to build powerful apps you need to get the underlying abstractions right.
Rich Hickey - Keynote - Step away from the computer
When was the last time you thought hard about a problem for an hour, day, month, year?
When have you confidently sat down to implement something for the first time?
The point he made by the end of the talk was something like: if you will think hard and well about a problem and write down what you learn then you will position yourself to confidently implement a solution on your first attempt
We commonly talk about the cheapest place to fix bugs is at development time. But, it is cheaper to fix bugs at design time. The biggest problems in software are ones of misconception. The developer has wrong ideas about the problem or the solution. Automated testing cannot provide a direct answer for the question of “whether this is a good idea?”
Analysis and Design
- not UML, not a process, not waterfall, step before “go do it!”
- identify and state the problem (verbally or in written form)
- critically assess proposed solutions
We should be solving problems, not adding features. Will the features solve a problem? Or will the features create a problem? Avoiding a problem is not the same as solving a problem.
Problem solving is a skill which can be learned and practiced. See “How to Solve It” Is it better to be good at problem solving or methodology X?
Understand the problem:
- facts, context, constraints (known and unknown), questions to find unknown unknowns, related problems
- understand previous attempts to solve the problem
- goal is to work incrementally ahead of the best known solution to the problem
- write this down
Be discerning:
- are there problems in your solution? If so, solve these.
- identify tradeoffs. Often stated tradeoffs just identify deficiencies in a solution. To properly understand tradeoffs it is necessary to have at least two alternative solutions in hand
- write this down
Focus is needed to think well:
- the computer is a distraction
- balls will be dropped while you are focusing deeply on a problem. This is part of the process.
- feed your background mind by thinking a lot about the problem with the waking mind. This will establish an agenda for the background mind to work.
- since the mind can only focus on ~7 items at a time, write down the aspects of a problem. This will allow you to quickly swap some of the items in and out of your mind. Maybe use pictures for this (not prescribing UML). Go over this repeatedly so you internalize it and can think about it long and hard.
- closing your eyes and removing other sensory inputs facilitates thinking well
- … wait for a solution to come
- since it is not always practical to wait, pipeline. Get several problems going at once. Switch tasks when you are stuck.
Once an idea for a solution has come:
- try it
- code it
- this should involve little typing
- since you have thought about it enough this should be confident coding
- feedback (e.g. from test results) is important but don’t lean on it
David Liebke - Concurrency to parallelism
The slides are very well down and contain much of the content of the presentation.
Concurrency is about managing access to shared state.
Parallelism is about making use of multiple processors.
pmap
- parallel meets lazy
- weakness - if there is a slow consumer then it degenerates (because the worker threads stop working ahead) [is that really a weakness if there is no consumer to use the results yet?]
- weakness - one slow processor can slow the entire process (maybe this is more of a theoretical problem?)
- it is more useful on long-running functions
fork join
- based on work of Doug Lea
- each worker has a double ended queue (deque) of work
- workers can “steal” from other workers when they are out of work. This provides some load balancing across threads.
- implemented in Clojure in fjvtree
- in order to make pvfilter on a pvreduce work it is necessary to make the reduce function associative and it needs to handle intermediate results (which might be in a different from that the leaf inputs)
Chas Emerick - Continuous deployment
slides
First principle - get your beautiful creating into the hands of your customers
Paths to deploying your apps:
- “system admin” - unrepeatable, manual, undocumented, high priesthood <- horrible
- “roll your own” automated system - typically the default solution, not really a solution, just yak-shaving
- community standards - this is where you want to be, stay close to the herd
Varying levels of community standards:
- vertically intergrated solution. This is the best, use it if you can. e.g. Google App Engine, Heroku, Azure
- more general tools. e.g. Chef or Puppet. But you need to use a non-Clojure language to express tasks.
- Clojure specific tools: jclouds (with Clojure wrappers), Pallet
Pallet
- pure Clojure
- uses jclouds
- similar to Chef
- “crates” (instead of recipes) are idempotent tasks that are “just” Clojure and totally composable
Don’t deploy to production on Jetty (ok to use it for development). Instead use Tomcat, JBoss, or Glassfish, etc.
Automate the build, test, and deploy process.
Stuart Halloway - Simplicity
slides
The word “simple” has been severely abused:
- it does not mean counting things (i.e. “this is simple because it has ”)
- it does not make sense for it to mean: how productive can a newbie be (e.g. kazoo vs violin). This is no way to build a career.
- it does not mean that something is familiar to us
- it does not mean that something is minimal (although this is often a valuable characteristic)
- it does not mean uncomplicated
So is the definition hopelessly subjective? No. It has an objective definition that should inform our designs and is fundamental to what we are trying to do.
Simple means: not compound
See “Studies in Words” by C.S. Lewis in which he talks about verbicide and how the meaning of words change over time. Specifically it examines the history of the meaning of the word “simple”
[… I had to leave early, so I missed the 2nd half of the presentation.]
Edit - Added link to Chas’ slides.
Edit - Added link to Stuart Halloway’s slides.
