Goodbye Paper of the Week

This is the last post in our “Paper of the Week” series. For more info, check out the introductory blog post.

We’ve decided to stop publishing Paper of the Week. Paper of the Week has been fun to write, but we don’t think it’s worth continuing given the limited response we’ve gotten and the amount of time it takes to put together. Instead, we’re going to focus on writing other things for our blog1.

For posterity (and because our blog doesn’t have categories), here are links to all past Papers of the Week collected in one place.

Thanks to everyone who submitted a paper, a Read Along, a suggestion, or a comment. We hope that Paper of the Week has been enjoyable, and maybe even a bit enlightening.

Happy reading!

  1. If you want to get email updates when we publish a new blog post or issue of Code Words, we have a new email list that you should subscribe to.

David albert 150
Tweet

A string of unexpected lengths

When you start learning to program, or working in a new language, it’s often suggested that you build a simple program like Battleship or Tic-tac-toe. The games' rules are well-defined and easy to grasp, and you only need to read and print text to get started. This frees you up to focus on the mechanics and ideas of the programming language you’re learning.

To create the game’s interface in the terminal, you end up doing a lot of string formatting: board layout, progress bars, announcements to the user. The length of a string is useful when formatting for terminals, since they usually use monospaced fonts. For example, while writing a game of Battleship in Python we might use the len() function explicitly for formatting math or implicitly in convenient built-in methods like center() to make exciting messages like the following:

>>> msg = 'battleship sunk!'
>>> len(msg)
16
>>> def underlined(msg):
...     return msg + '\n' + '-' * len(msg)
...
>>> print underlined(msg)
battleship sunk!
----------------
>>> print msg.center(30, '*')
*******battleship sunk!*******

However, the code above won’t always work as we expect because the len() of text isn’t necessarily the same as its width when displayed in a terminal. Let’s explore three ways these numbers can differ.

Multiple bytes for one character

Byte strings (known as “strings” in Python 2) have formatting methods like center() which assume that the displayed width of a string is equal to the number of bytes it contains. But this assumption doesn’t always hold! The single visible character Ä might be encoded as several bytes in a source file or terminal.

>>> shipname = 'Ägir'
>>> shipname
'\xc3\x84gir'
>>> len(shipname)
5
>>> print shipname.center(10, '=')
==Ägir===
>>> print shipname + '\n' + '-' * len(shipname)
Ägir
-----

The number of bytes in this byte string doesn’t match the number of characters so built-in formatting operations don’t behave correctly.

Fortunately, using Unicode strings instead of byte strings solves this problem because they usually report a length equal to the number of Unicode code points they contain.1

>>> shipname = u'Ägir'
>>> len(shipname)
4
>>> shipname.center(10, u'=')
u'===\xc4gir==='
>>> print shipname.center(16, u'*')
===Ägir===
>>> print shipname + '\n' + '-' * len(shipname)
Ägir
----

ANSI escape code formatting

ANSI escape codes let us format text by writing bytes like '\x1b[31m' to start writing in red, and '\x1b[39m' to stop. If we build a string containing these sequences, the calculated length of our string won’t match its displayed width in a terminal:

>>> s = '\x1b[31mhit!\x1b[0m'
>>> print s
hit!
>>> len(s)
13
>>> print s + '\n' + '-' * len(s)
hit!
-------------
>>> print s.center(14, '*')
hit!*

The colored string reports a length larger than its displayed width, causing problems for built-in text-alignment methods. Fortunately, there are several Python libraries that make it easier to work with colored string-like objects that don’t include formatting characters in their length calculations.

Clint’s colored strings have formatting methods that produce the output you expect:

>>> from clint.textui.colored import green
>>> len(green(u'ship'))
4
>>> green(u'ship').center(10)
<GREEN-string: u'   ship   '>
>>> print green(u'ship').center(10)
   ship   

but this no longer works once two colored strings are combined into a new colored string:

>>> from clint.textui.colored import blue, green
>>> len(green('ship') + blue('ocean'))
39
>>> green('ship') + blue('ocean')
'\x1b[31m\x1b[22mship\x1b[39m\x1b[34m\x1b[22mocean\x1b[39m'
>>> print (green('ship') + blue('ocean')).center(10, '*')
shipocean

My own attempt at solving this problem uses smart string objects which know how to concatenate:

>>> from curtsies.fmtfuncs import green, blue
>>> len(green(u'ship'))
4
>>> green(u'ship').center(10)
green("   ship   ")
>>> print green(u'ship').center(10)
   ship   
>>> s = green(u'ship') + blue(u'ocean')
>>> len(s)
9
>>> print s.center(13, '*')
**shipocean**

but doesn’t correctly implement every formatting method yet: above, **shipocean** has lost its color information because a fallback implementation of center() was used.2

The Unicode jungle

Formatting methods of Python Unicode strings like center() assume that the display width of a string is equal to its character count. But this assumption doesn’t always hold!

What if we use fullwidth Unicode characters?

>>> battleship = u'扶桑'
>>> len(battleship)
2
>>> print battleship + '\n' + '-' * len(battleship)
扶桑
--

What about multiple Unicode code points that combine to display a single character?3

>>> battleship = u'Fuso\u0304'
>>> print battleship
Fusō
>>> len(battleship)
5
>>> print battleship.center(6, u'*')
*Fusō

The width of a Unicode string differs from the number of characters in it. Fortunately, we can use the POSIX standard function wcswidth to calculate the display width of a Unicode string. We can use this function to rebuild our basic formatting functionality.4

>>> from wcwidth import wcswidth
>>> wcswidth(battleship)
4
>>> def center(s, n, fillchar=' '):
...     pad = max(0, n - wcswidth(s))
...     lpad, rpad = (pad + 1) // 2, pad // 2
...     return lpad * fillchar + s + rpad * fillchar
...
>>> print center(c, 6, '*')
*Fusō*
  1. Unfortunately, for versions of Python earlier than 3.3 it’s still possible that the len() of a Unicode character like u'\U00010123' will be 2 if your Python was built to use the “narrow” internal representation of Unicode. You can check this with sys.maxunicode - if it’s a number less than the total number of Unicode code points, some Unicode characters are going to have a len() other than 1.

  2. Want to fix this? Pull requests are welcome! The fix would be pretty similar to the fix for this issue about .ljust and .rjust.

  3. The Unicode spec calls this an extended grapheme cluster. Interestingly, the Character class in the Swift programming language represents an extended grapheme cluster and may be composed of multiple Unicode code points.

  4. Here we’re using a pure Python implementation for compatibility and readability.

Thomas ballinger 150
Tweet

A day at Hacker School

Applicants commonly ask what a typical day at Hacker School is like. Hacker School is very self-directed, so there isn’t a simple answer. People allocate their time differently, both person by person and day by day, but there are a few buckets into which most activities fall.

The primary constant is checkins. The official start of the Hacker School day is 10:30am1, when we have morning checkins. We do checkins in groups of four to six people, and the groups change every week.2 Checkins serve several purposes. First, they help make Hacker School an intentional community by delineating a clear start of the day. Second, they provide a form of lightweight social accountability: They help folks set goals for themselves and stick to them. Third, they help people hear about what others are working on, and possibly pair up. Lastly, they provide an easy way to ask for help. Checkins usually last between five and fifteen minutes.

On Thursdays, we have weekly presentations at the end of the day. Presentations are entirely optional: You can choose whether or not to attend, and you are welcome to present or just watch. Presenters also choose how long they want to talk for, usually between one and five minutes. Attending is a fun way to hear what others are working on, find potential collaborators, and be exposed to a wide range of ideas and programming topics. Presenting is a great way to practice and improve talking about code and technical ideas to a friendly audience of peers.

On Monday nights, we have dinner and a technical talk. The talks are typically given by residents or other invited speakers, and are meant to be accessible and interesting to a wide range of programmers (i.e., they don’t assume any specific domain expertise). Our format is a 20 to 30-minute talk with no questions, followed by a 90-second break when it’s socially acceptable to leave, followed by an extended Q&A period for those who choose to stick around. We don’t have questions during our talks because we want them to remain short. We have a break before Q&A begins because we want people to have a polite opportunity to leave if they’d like.

Checkins, Monday night dinners, and Thursday presentations only make up a small part of Hacker School. Most of Hacker School is unstructured by design: We believe in giving people the freedom to use their time and opt into the structure that will be most effective for them. What people choose to do at Hacker School is typically a combination of the following:

Pairing. We encourage people to spend at least some of their time pair programming. Pairing is one of the best and most common ways to take advantage of the incredible group of people Hacker School brings together.

Working independently. While pairing is great, most Hacker Schoolers also choose to spend a good chunk of their time coding or studying on their own.

Getting or giving code review. Code review is an excellent way to get feedback on your code. We encourage people to get (and give!) regular code reviews as an effective way to write better code and track their own progress.

Attending or running short workshops or seminars. Current Hacker Schoolers, alumni, facilitators, and residents all occasionally run small workshops and seminars. Some of these are organized far in advance and many are fully ad hoc. Recent sessions have included a deep dive into how Git’s internals work, operating systems research, and how to effectively read Haskell code.

Participating in book or other small working groups. Hacker Schoolers organize one-off and ongoing groups for all kinds of things. A perennial favorite are SICP work groups, where a handful of people agree to meet regularly (e.g., “Tuesdays and Thursdays at 11am”) to go over their solutions to the book’s exercises.

Doing office hours. Hacker Schoolers can always grab a free facilitator to discuss their goals, code, or anything else. Additionally, we have a office hour system so people can book time with facilitators in advance.

Looking back over this post, it successfully conveys the mechanics of Hacker School, but fails to capture its essence. That’s because many of the best parts of Hacker School are unplanned, or at least not planned directly. The value of Hacker School is ultimately an emergent result of its base conditions: Bringing together a diverse group of friendly, intellectually curious people, providing a safe and resource-rich environment, and giving them the freedom to work on and learn about what they’re interested in. The great things this leads to are serendipitous and too numerous to enumerate.

  1. Hacker Schoolers get 24/7 access to the space, and so some people come in much earlier, though most arrive between 10 and 10:30am. A 10:30am start time is a compromise, since past Hacker Schoolers have said their ideal start times would range from 6am to 12pm.

  2. We’ve experimented with many variations for how to form and rotate groups. The current incarnation is this: Each week people choose a group from a sign-up sheet. Half of the groups are topic-based, and the first Hacker Schooler to sign up for one of those groups can set a topic (e.g., “Ruby” or “Distributed systems”). The other half of the groups are “random” – i.e., they don’t have any specific topic.

Nick bergson shilcock 150
Tweet

Paper of the Week: Compositing Digital Images

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is Compositing Digital Images by Thomas Porter and Tom Duff (of Duff’s device). It was published in the July 1984 issue of Computer Graphics. Porter and Duff had worked together at Lucasfilm’s Graphics Group (which later became Pixar), though Duff was working at Bell Labs at the time of the paper’s publication.

Compositing Digital Images is a very cool piece of history that describes a method of compositing—combining a number of source images into one output image—that we still use virtually unchanged 30 years later. The paper introduces two ideas: an alpha channel for storing per-pixel opacity information (the A in RGBA) and a set of operations for combining two images that Porter and Duff call a compositing algebra.

Using an alpha channel to track opacity was not a new idea when the paper was written, but this may have been the first time the idea was presented to the wider world. The compositing algebra, on the other hand, was brand new and is especially elegant.

This paper was my first rigorous introductions into the world of computer graphics, but reading it felt familiar. Its ideas seem to pop up again and again, whether I’m building websites, programming simple games, or messing around in Photoshop. If you’re looking for an introduction to computer graphics, this is a great place to start.

Here’s the abstract:

Most computer graphics pictures have been computed all at once, so that the rendering program takes care of all computations relating to the overlap of objects. There are several applications, however, where elements must be rendered separately, relying on compositing techniques for the anti-aliased accumulation of the full image. This paper presents the case for four-channel pictures, demonstrating that a matte component can be computed similarly to the color channels. The paper discusses guidelines for the generation of elements and the arithmetic for their arbitrary compositing.

Read Along

Read Along is a way for you to participate in Paper of the Week. All you have to do is read the paper, make something small in response (code or prose), and email us a link to what you made by noon Eastern Time next Monday.

There were no Read Along submissions to last week’s paper. If you’re looking for a project for this week, we think a minimal JavaScript implementation of the compositing algebra described in this paper could be a lot of fun.

Happy reading!

David albert 150
Tweet

Paper of the Week: Literate Programming

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is Literate Programming by Donald Knuth. It was published in the May 1984 issue of The Computer Journal.

In this paper, Knuth introduces literate programming, a programming style where computer programs are written like essays with prose interleaved with code. Literate programming probably hasn’t caught on like Knuth had hoped when he wrote this, but it has a small, extremely dedicated following.

Literate Programming was submitted by Hacker Schooler Waldemar Quevedo who shared the following:

I like this paper a lot not only because it is written by Knuth and because it has one of the coolest quotable opening phrases I’ve seen: (“I believe that the time is ripe for significantly better…”) but also because it really shows what practicing literate programming does to you if you are to practice it.

In this paper, Knuth announces (maybe a bit too prematurely) about better times which are coming to the practices of software development based on his experiences of using the WEB system to craft his software.

By the end of the writing, Knuth shares how literate programming has consumed most of his workflow to the point that he sees little benefit of not using literate programming for any of his development.

As an Org mode user, I was surprised how literate programming with Org Babel gradually became the development environment I felt the most comfortable working with.

The abstract below doesn’t do justice to Knuth’s enthusiasm for literate programming, nor the humor and clarity with which the paper is written. If you’re not convinced, spend 3 minutes reading through the introduction. You won’t be disappointed.

The author and his associates have been experimenting for the past several years with a programming language and documentation system called WEB. This paper presents WEB by example, and discusses why the new system appears to be an improvement over previous ones.

Read Along

Read Along is a way for you to participate in Paper of the Week. All you have to do is read the paper, make something small in response (code or prose), and email us a link to what you made by noon Eastern Time next Monday.

There were no Read Along submissions to last week’s paper. We’re looking forward to seeing the literate programs you write this week!

Happy reading!

David albert 150
Tweet

Paper of the Week: The Evolution of the Unix Time-sharing System

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is The Evolution of the Unix Time-sharing System by Dennis Ritchie. It was presented at the 1979 Language Design and Programming Methodology conference in Sydney, Australia.

This paper describes the early history of Unix, which was created in 1969. It’s fun to see what’s been the same since the beginning (IO redirection), what’s changed (there weren’t any path names!), and how things got to be the way they are today. As supplemental reading, you may also enjoy Ritchie’s short writeup of Space Travel, the game for the PDP-7 that preceded Unix. If you’re feeling ambitious this weekend, building a modern version of Space Travel might be a fun exercise (and a great Read Along submission).

The Evolution of the Unix Time-sharing System was recommended by Hacker School resident Michael Lee.

Here’s the abstract:

This paper presents a brief history of the early development of the Unix operating system. It concentrates on the evolution of the file system, the process-control mechanism, and the idea of pipelined commands. Some attention is paid to social conditions during the development of the system.

Read Along

Read Along is a way for you to participate in Paper of the Week. All you have to do is read the paper, make something small in response (code or prose), and email us a link to what you made by noon Eastern Time next Monday.

Hacker School alum Oskar Thorén submitted a tweet that he claims gets close to the essence of Error Detecting and Error Correcting Codes. Thanks Oskar!

Happy reading!

David albert 150
Tweet

Paper of the Week: Xen and the Art of Virtualization

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

After an extended New Year’s break, Paper of the Week is back! This week’s paper is Xen and the Art of Virtualization by Paul Barham from Microsoft Research, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Ian Pratt, and Andrew Warfield from the University of Cambridge Computer Laboratory, and Rolf Neugebauer from Intel Research. It was presented at the 2003 ACM Symposium on Operating Systems Principles.

This paper introduces Xen, a virtual machine monitor, or hypervisor, that allows you simultaneously and efficiently run multiple operating systems on a single piece of hardware. Twelve years after this paper was published, Xen is now a critical piece of Internet infrastructure and powers many services, including Amazon EC2. Even this blog is run on top of Xen.

Xen and the Art of Virtualization was recommended by Hacker School resident Michael Lee, who shared the following:

Part of what I like about this paper comes from impact: Xen is wildly successful and deployed all over the place. Part of it comes from the clever use of ring 1 as a way to deprivilege the guest OS. And part of it comes from the great performance they’re able to get out of the system. In my opinion, one of the key insights is realizing when it is alright to ask the developer to modify their system. In the case of Xen, they only ask the OS developer to make modifications and ensure compatibility for the wider legacy software. In doing so, they were able to attain much better performance, side step many of the difficulties presented when virtualizing x86, and even make some of the hardware interfaces cleaner for the OS developers to help ease the transition.

Here’s the abstract:

Numerous systems have been designed which use virtualization to subdivide the ample resources of a modern computer. Some require specialized hardware, or cannot support commodity operating systems. Some target 100% binary compatibility at the expense of performance. Others sacrifice security or functionality for speed. Few offer resource isolation or performance guarantees; most provide only best-effort provisioning, risking denial of service.

This paper presents Xen, an x86 virtual machine monitor which allows multiple commodity operating systems to share conventional hardware in a safe and resource managed fashion, but without sacrificing either performance or functionality. This is achieved by providing an idealized virtual machine abstraction to which operating systems such as Linux, BSD and Windows XP, can be ported with minimal effort.

Our design is targeted at hosting up to 100 virtual machine instances simultaneously on a modern server. The virtualization approach taken by Xen is extremely efficient: we allow operating systems such as Linux and Windows XP to be hosted simultaneously for a negligible performance overhead — at most a few percent compared with the unvirtualized case. We considerably outperform competing commercial and freely available solutions in a range of microbenchmarks and system-wide tests.

Read Along

Read Along is a way for you to participate in Paper of the Week. All you have to do is read the paper, make something small in response (code or prose), and email us a link to what you made by noon Eastern Time next Monday.

The last Paper of the Week was Open, extensible object models by Ian Piumarta and Alessandro Warth. Here are this week’s Read Alongs:

Happy reading!

David albert 150
Tweet

A small step in a new direction

As we recently announced, we’re thrilled to have Chris Granger and Jamie Brandon coming to Hacker School this February for a two-week residency. Chris is the co-founder and CEO of Kodowa, and Jamie is a Hacker School alum who works at Kodowa with Chris.

We are especially excited because this will be an experiment for us: Chris and Jamie’s residency will be different from those in the past because they will be coming with an explicit focus on sharing and continuing their work and research on Eve with Hacker Schoolers.

Eve is an ambitious project to make computation widely accessible by re-imagining how we interact with computers. As such, it hits on fundamental and exciting questions of computer science and programming. Like Chris and Jamie’s previous project, Light Table, Eve will be released as open source.

Eve is still early in its development, and has many open research questions. Jamie tells us these include everything from technical problems, like query planning and incremental view updates, to people problems, like managing the system’s learning curve. Beyond these questions, Chris and Jamie are interested in collaborating with Hacker Schoolers on anything that involves rethinking part of the programming process. You can read more in the Eve dev diary.

This residency will be a small step in a direction we hope to ultimately go much further: Supporting and encouraging programming research at Hacker School.

Nick bergson shilcock 150
Tweet

Announcing eight new residents for 2015

We’re excited to announce eight new Hacker School residents for 2015! If you’d like to work with any of these residents, apply to Hacker School for a 2015 batch.

Mel Chua Perennial resident Mel Chua will be in residence from 1/7 – 1/9. Mel is a contagiously enthusiastic hacker, writer, and educator. Her Engineering Education research at Purdue focuses on how hackers learn and how engineers are taught. You may know her from the Ada Initiative, Red Hat’s Community Leadership Team, Fedora, Sugar Labs, One Laptop Per Child, the MIT Media Lab, Design Continuum, Appropedia, or OpenPlans, among other places.

Scott Vokes Scott Vokes will be in residence the week of 1/26 – 1/29. He is a consultant at Atomic Object, where he builds embedded and distributed systems. Outside of work, his research interests include information retrieval—recent open source contributions include a data compression library for hard real-time systems, testing tools for C, and a content-addressable store. Scott is also the creator of @ghost_things, a byproduct of his love for wordplay.

Sam Tobin-Hochstadt Sam Tobin-Hochstadt will be in residence the week of 2/2 – 2/6. Sam is an Assistant Professor in the School of Informatics and Computing at Indiana University. He has worked on dynamic languages, type systems, module systems, and metaprogramming, including creating the Typed Racket system and popularizing the phrase “scripts to programs.” He is a member of the ECMA TC39 working group responsible for standardizing JavaScript, where he co-designed the module system for ES6, the next version of JavaScript. He received his PhD in 2010 from Northeastern University under Matthias Felleisen.

Chris Granger Chris Granger and Jamie Brandon will be in residence for the weeks of 2/23 – 3/6. Chris grew up as part of the Nintendo generation, having learned the parts of a computer at the age of two and later learning numbers and colors from a Sesame Street game on the NES. He started programming at the age of ten and took his first paid development gig at 17. Since then he’s built websites large and small, written frameworks and libraries used by thousands, taught developers around the world, and helped envision the future of development at Microsoft. These days, he’s the co-founder and CEO of Kodowa, where they built the next generation code editor Light Table and now Eve, a new vision for putting computation in the hands of everyone.

Jamie Brandon Jamie spent years travelling the world as a consulting computer scientist. He has seen everything from finance and gaming to the depths of corporate IT and is now pretty sure we are doing it wrong. Currently he is working on rethinking programming with Eve and hoping to do it at least slightly less wrong this time around. Jamie’s residency also marks the first time a Hacker School alumnus has returned as a resident, and we’re excited to have him back!

Chris and Jamie’s residency will be a bit different from our others. You can learn more about their plans for their residency and their project Eve in this blog post!

Patrick Dubroy Patrick Dubroy will be in residence for the week of 3/9 – 3/12. Pat is a programmer and art school dropout.He currently works with the Viewpoints Research Institute and the Communications Design Group at SAP Labs, developing languages and tools to make programming better. Previously, he worked at Google as a software engineer on Chrome and Android, and at BumpTop as a programmer and interaction designer. Originally from Canada, he now lives in Munich (and yes, he owns a pair of Lederhosn).

Neha Narula Neha Narula will be in residence for one week in April 2015. She works on distributed systems and makes multicore databases go faster. She has worked on a system for executing queries on a sharded database called Dixie, an in-memory cache with materialized views called Pequod, and a multicore database that leverages commutativity called Doppel. In a previous life she was a software engineer at Google where she worked on Google Shopping, Blobstore, and Native Client.

R0ml Robert Lefkowitz (a.k.a. r0ml) will be returning to Hacker School for another residency for the week of 4/6 – 4/9. R0ml is a programming language enthusiast who enjoys obscure programming languages. He is a collector of programming techniques for improving clarity, increasing reliability, and maximizing brevity. He is a frequent speaker on the effects of the late Middle Ages and early Renaissance on the art of programming. His first programming project was on an Olivetti-Underwood Programma 101. He is an ACM Distinguished Engineer. Photo by Amanda Thomas.

Rachel vincent 150
Tweet

Four failure modes of Hacker Schoolers

New Hacker Schoolers are often surprised by just how unstructured Hacker School is. During Hacker School, participants are free to get better as programmers using whatever strategies work best for them, covering whatever topics they’re most interested in. This is a very different environment from most schools and workplaces, and it often takes time to adjust to.

Hacker Schoolers are responsible for their own learning experiences. While facilitators are available to support Hacker Schoolers and help them make the most of this unusual environment, we never coerce Hacker Schoolers into projects, topics, or strategies that aren’t effective or enjoyable for them. Since ultimate responsibility to succeed lies with Hacker Schoolers, they’re also free to fail—to make little progress or be unproductive.

This post discusses four of the most common pitfalls we’ve seen Hacker Schoolers struggle with, particularly at the beginning of their batches. The goal is to help Hacker Schoolers identify when they’re in one of these failure modes and quickly fix the problem. The post doesn’t discuss solutions to these problems. For one thing, the solutions are more individualized than the problems. More importantly, we believe that it’s useful just to help people recognize and name something they may be struggling with.

Not being public

The first failure mode is hiding, or not being public. By “public” I mean being visible to others in some way. This includes pairing, blogging, talking at weekly presentations, giving a small-group seminar, conducting or receiving code review, and talking on our internal chat system. Nothing at Hacker School forces you to do any of these things, but your experience will be much better if you do them. Being public allows you to learn from your peers, to discover your blind spots, and to leverage others' experience to move faster. Being public is a prerequisite for getting feedback, and feedback is a great way to grow.

Too much task-switching

The second failure mode is too much task-switching. It’s hard to decide what to work on. This is something that people who’ve had traditional educational experiences or traditional jobs tend to really struggle with at first. As a student, you make only a handful of decisions per semester—which classes to take, maybe which topic to write a paper about, and that’s it. All the rest is time-management, deciding which of your mandatory tasks you should do in which timeslot. Similarly, as an employee of a hierarchical company, your boss might set your priorities and your deadlines. Going from this model to one where you decide which things are most important in the absence of external deadlines is a significant change. It takes practice to master this skill.

This failure case emerges when Hacker Schoolers refuse to prioritize at all, or worry excessively about their prioritization. The results of these two attitudes are the same: constant task-switching between everything that seems even minutely important.

The Hacker Schooler who won’t prioritize at all won’t say “no” to a project—or even “not now.” They end up with a day cut into one-hour chunks of study groups, reading, pairing, following tutorials, and working independently, all on different topics and in different languages. This is a very difficult way to make progress. Most people underestimate the cost they pay from switching contexts.

The Hacker Schooler who worries too much about their prioritization wastes cycles. They tend to doubt their prioritization the most at exactly the moment that their project gets hard. Because they’re not committed to the project, they’re less likely to seek help, more likely to tab over to Twitter when they’re stuck, and more likely to give up without learning anything interesting.

Impostor syndrome

The third failure mode, impostor syndrome, is a bit more tricky. There are three problems with impostor syndrome at Hacker School. First, people who would be great Hacker Schoolers don’t apply because they don’t think they’ll get in. Second, Hacker Schoolers don’t try hard things, because they’re afraid they’ll fail at them and everyone will know what frauds they are. Third, Hacker Schoolers don’t publish their work or otherwise aren’t willing to be public.

Impostor syndrome is scary. It’s particularly harmful at Hacker School because Hacker School relies on the individual to push themselves and to take risks. As mentioned above, the Hacker School faculty won’t coerce people into tasks or projects. If you stay in your comfort zone and only work on things you know you can succeed at, you’re removing much of the opportunity for exceptional personal growth.

The flip side of this is that Hacker School has no consequences for “failing” at a project. You may work on something for several weeks before deciding it’s impossible, not consistent with your learning goals, or just not useful for you anymore. There’s little downside to not finishing something at Hacker School. This is very different than most jobs, college courses, and PhD research projects, and we encourage Hacker Schoolers to take full advantage of this environment by attempting things that they aren’t sure they’ll be able to do.

Job-hunting / extrinsic motivation

A final thing that Hacker Schoolers struggle with is job-hunting. Job-hunting is not only time-consuming and distracting, it also shifts Hacker Schoolers' motivations. Hacker School relies intensely—even to a fault—on intrinsic motivation. Job-hunting, by contrast, encourages Hacker Schoolers to start thinking about what parts of their work would be considered impressive, or how they can make their projects seem shiny. People often end up plodding through projects they don’t enjoy because they think someone, somewhere, will be impressed. In reality, it’s very hard to do great work on something you hate.

Earlier in Hacker School’s history, we told people not to think about jobs at all during Hacker School. This wasn’t a good idea either, as discussed in Mistakes We’ve Made. Now, we try to help Hacker Schoolers strike a balance by having separate, optional interview preparation sessions outside of Hacker School’s official hours.


If you’re a Hacker Schooler who’s wrestling with one or more of these failure modes during your batch, you should know that it’s common to feel this way, and it’s also common to feel more comfortable as time passes and you experiment with different strategies. To use Kathy Sierra’s construction, these problems are typical and temporary. Quickly recognizing when you’re in one of these failure modes and changing your habits will help you make the most of the opportunity you have before you.

Allison is a Hacker School alum and was a facilitator for over two years. She’s now a software engineer at Dropbox.

Allison kaptur 150
Tweet

Paper of the Week: Open, extensible object models

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is Open, extensible object models by Ian Piumarta and Alessandro Warth and published in 2008 in the book Self-Sustaining Systems.

This paper describes a small, flexible object model that implements method lookup using a prototype-like system similar to JavaScript. It’s more interesting than it sounds though: method lookup itself is defined in terms of sending messages to objects, so you can change how method lookup works without having to change the underlying implementation. This means that you can implement prototypes, classes, multiple inheritance, traits, and anything else you can dream up, in one program without digging into the internals of the language.

What makes this even cooler is how small it is. To support all this, you only need three object types and five methods, making this as close to a Maxwell’s equations for objects as I’ve seen. There’s even a standalone implementation in 230 lines of C at the end of the paper.

Here’s the abstract:

Programming languages often hide their implementation at a level of abstraction that is inaccessible to programmers. Decisions and tradeoffs made by the language designer at this level (single vs. multiple inheritance, mixins vs. Traits, dynamic dispatch vs. static case analysis, etc.) cannot be repaired easily by the programmer when they prove inconvenient or inadequate. The artificial distinction between implementation language and end-user language can be eliminated by implementing the language using only end-user objects and messages, making the implementation accessible for arbitrary modification by programmers. We show that three object types and five methods are sufficient to bootstrap an extensible object model and messaging semantics that are described entirely in terms of those same objects and messages. Raising the implementation to the programmers' level lets them design and control their own implementation mechanisms in which to express concise solutions and frees the original language designer from ever having to say “I’m sorry.”

Read Along

Read Along is a way for you to participate in Paper of the Week. All you have to do is read the paper, make something small in response (code or prose), and email us a link to what you made by noon Eastern Time next Monday.

This week, Thomas Smith submitted a partial implementation of Rob Pike’s Structural Regular Expressions. Thanks, Thomas!

Happy reading!

David albert 150
Tweet

Negative comments

Here’s a theory for why there are so many negative comments on the Internet: We train people to write them.

By negative comments, I don’t mean purely nasty comments like those on YouTube. I mean the comments that are common on Hacker News. These comments are reasonably well-written but boil down to “here is why X is wrong” in the best cases and “X is stupid” in the worst. These comments are negative in the traditional sense of the word: They aim to deny a point, or disagree.1

Learning how to disagree with and challenge ideas was a core part of many of my college classes. To this day, when I read something my mind immediately starts searching for contradictions and errors, trying to find everything it can to challenge. This is habitual for me, and is more a reflex than a conscious decision.

Indeed, Western culture trains us to disagree as part of learning to be critical thinkers. Critical thinking in its current form was first championed by the great progressive educator John Dewey, but its roots date all the way back to the Greeks, when Socrates spoke about the importance of critical self-examination to find errors in our thinking.

There’s a legitimate danger to uncritically consuming ideas floating around us. Without a critical eye, it’s easy to start believing things we’re taught unthinkingly, and I am a huge believer in the importance of critical thinking skills. But I fear emphasis has shifted from critically reflecting on and examining our own beliefs to simply criticizing and pointing out errors in other people’s work.

This shift is understandable because it’s so much easier to challenge other people’s thinking than our own. Finding errors in our own thinking is hard: First because it means discovering our personal blind spots, and second because it means admitting we’re wrong. In contrast, writing negative comments feels good: It exercises our critical thinking skills without challenging anything we hold dear.

The cynical explanation for this is that people write negative comments to show off how clever they are or how much they know. But I don’t think that’s enough to explain how dedicated many commenters are to posting negative feedback. Instead, I think people do it because they believe it’s the right thing. Our cultural obsession with critical thinking compels us to point out errors when we perceive them; errors are injustices that we must right.2 Someone is wrong on the internet, and duty calls.

But pointing out all the places other people are wrong rarely teaches us anything. So next time you read something, try this: Instead of looking for the parts you can prove are false, try to find pieces you can learn from.

Incidentally, I think this in part explains why the Hacker School community is so much more positive than the world at large: People come here firstly to learn new things, not to dispute them. This suggests an interesting question: Could you build a site like Hacker News with a community focused on learning above all else?

  1. Negative comes from the Latin negativus, meaning “that which denies.” A negative comment is a comment that denies (or disagrees with) a part of what it’s responding to.

  2. Some errors are injustices and should be corrected, but most are not. Worse, many “corrections” in comments are themselves wrong, and suggest a preference for finding things to dispute over understanding what an author is trying to say.

Nick bergson shilcock 150
Tweet

Michael Lee is a last minute resident

Michael Lee

We’re excited to have Michael Lee in residency at Hacker School this week!

Michael is a fifth year Ph.D. student at UT Austin studying security and operating systems. He works on improving and changing the security properties of commodity operating systems and research into creating practical crypto protocols and systems. He has also collaborated with researchers at Columbia University and Sandia National Labs.

Michael has already given a seminar on Anon-Pass, an anonymous subscription system, and is in the middle of giving another on the guts of the Linux kernel with an eye towards security.

If you want to program with residents like Michael, you should apply to Hacker School. We’ll be announcing more residents in the next few weeks.

David albert 150
Tweet

Paper of the Week: Structural Regular Expressions

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is Structural Regular Expressions by Rob Pike, one of the creators of the Go programming language and the Plan 9 operating system. It was published in Proceedings of the EUUG Spring 1987 Conference.

This paper is all about picking the right abstraction. Sometimes a small change to the abstraction you’re using can dramatically simplify your code, or allow you to express a larger set of ideas. Structural Regular Expressions has a particularly good example of this. With one small change to how traditional regular expressions match, Pike shows how a number of common tasks become easier to express. The paper is accessible and short, and the examples are elegant and easy to read.

Structural Regular Expressions was suggested by Hacker Schooler Travis McDemus, who shared the following:

I benefited in a few ways from reading this concise, accessible paper: 1) Observing some of the biases inherent throughout Unix. 2) Greater understanding of regular expressions as a whole (which can help with daily practical usage of vim, ed, sed, awk). 3) Understanding some of the philosophy that went into designing Plan 9’s tools, golang, etc.

Here’s the abstract:

The current UNIX® text processing tools are weakened by the built-in concept of a line. There is a simple notation that can describe the ‘shape’ of files when the typical array-of-lines picture is inadequate. That notation is regular expressions. Using regular expressions to describe the structure in addition to the contents of files has interesting applications, and yields elegant methods for dealing with some problems the current tools handle clumsily. When operations using these expressions are composed, the result is reminiscent of shell pipelines.

Read Along

Read Along is a way for you to participate in Paper of the Week. If you want to take part, all you have to do is read the paper, make something small in response (code or prose), and email us a link to what you made by noon Eastern Time next Monday.

We didn’t get any Read Along submissions for Statecharts: A Visual Formalism for Complex Systems :(. We’re looking forward to your submissions for this week’s paper.

If you’re looking for even more things to read, check out the first issue of Code Words, our new quarterly publication about programming.

Happy reading!

David albert 150
Tweet

Introducing Code Words

The Hacker School community has its share of prolific writers. Shortly after I started working here and reading everyone’s blogs, I realized that there was an opportunity to share the work of Hacker Schoolers, residents and alumni with a broader community of programmers. When I mentioned this to Sonali, she put me in touch with Sean Murphy (W'14), who had had much the same idea a few months earlier.

After a slew of meetings to figure out what Code Words was going to be, we came to a consensus: we wanted to create a quarterly journal to collect the work of folks in our community and share the joyful approach to programming and learning that typifies Hacker Schoolers. Like Hacker School itself, we want to make Code Words accessible and useful to both new and seasoned programmers.

We decided to see if there was any greater interest in the project, and the emphatic response to a call for submissions and editors confirmed that there was. With help and sage advice from several members of our community including Peter Seibel, Sumana Harihareswara, and Sasha Laundy, we put together contracts for our writers (we’re using the Creative Commons CC BY-NC-SA 4.0 license), a style guide, an editor’s manual and an editorial calendar. We edited the first issue on GitHub.

The process was not without hiccups. And in typical Hacker School fashion, there was also plenty of improvisation. But thanks to the work of the Hacker School community, some great design and development work from Sonali and Zach, and one of the greatest 404 pages of all time thanks to Alan and Dave, Issue 1 of Code Words is finally here.

We’re excited to share it with you, and to hear what you think.

Rachel vincent 150
Tweet

23 Hacker Schoolers and Residents on their experiences at Hacker School

One of the best ways to get a sense of what Hacker School is actually like is to read first-hand accounts of people who have done it. Fortunately, many alumni and residents have blogged about their experiences at Hacker School. Unfortunately, these posts are scattered across the web and can be hard to find. We’ve collected many of these blog posts here.

Posts by Hacker Schoolers

The posts below were written by current Hacker Schoolers or alumni, either during or after their batches. This is just a sampling of the posts folks have written about their experiences at Hacker School, and is presented in no particular order and in no way intended to be comprehensive.

Posts by residents

The posts below were written by residents, who visit Hacker School in one- or two-week stints.

Nick bergson shilcock 150
Tweet

Paper of the Week: Statecharts: A Visual Formalism for Complex Systems

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is Statecharts: A Visual Formalism for Complex Systems by David Harel, a professor of computer science and applied mathematics at the Weizmann Institute of Science. It was published in the June 1987 issue of Science of Computer Programming.

This week’s paper was submitted by Hacker School resident Kevin Lynagh, who shared the following:

Harel’s Statecharts are a visual formalism for specifying the behavior of systems. They’re basically finite state machines, but with notions of hierarchy and concurrency.

The paper is very readable: Harel reverse engineers his digital wristwatch and draws the statechart specifying its behavior. It’s extremely fruitful to internalize the larger idea: That you can (and should!) think rigorously about the desired behavior of an artifact before diving straight into the code.

Here’s the abstract from the paper:

We present a broad extension of the conventional formalism of state machines and state diagrams, that is relevant to the specification and design of complex discrete-event systems, such as multi-computer real-time systems, communication protocols and digital control units. Our diagrams, which we call statecharts, extend conventional state-transition diagrams with essentially three elements, dealing, respectively, with the notions of hierarchy, concurrency and communication. These transform the language of state diagrams into a highly structured and economical description language. Statecharts are thus compact and expressive—small diagrams can express complex behavior—as well as compositional and modular. When coupled with the capabilities of computerized graphics, statecharts enable viewing the description at different levels of detail, and make even very large specifications manageable and comprehensible. In fact, we intend to demonstrate here that statecharts counter many of the objections raised against conventional state diagrams, and thus appear to render specification by diagrams an attractive and plausible approach. Statecharts can be used either as a stand-alone behavioral description or as part of a more general design methodology that deals also with the system’s other aspects, such as functional decomposition and data-flow specification. We also discuss some practical experience that was gained over the last three years in applying the statechart formalism to the specification of a particularly complex system.

Read Along

Read Along is a way for you to participate in Paper of the Week. If you want to take part, all you have to do is read the paper, make something small in response (code or prose), and email us a link to what you made by noon Eastern Time next Monday.

Sadly, we didn’t receive any Read Along submissions for Notation as a Tool of Thought. We’re looking forward to your submissions for this week’s paper.

Happy reading!

David albert 150
Tweet

Paper of the Week: Notation as a Tool of Thought

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is Notation as a Tool of Thought by Kenneth Iverson. Iverson originally presented this paper as his Turing Award Lecture in 1979. It was published in the August 1980 issue of Communications of the ACM. The paper serves both as a primer on Iverson’s ideas about notation and an introduction to the APL programming language.

If you’re living in the United States, this is a perfect paper to read on your trip home for Thanksgiving. If you’re not living in the US, it’s a perfect paper to read any time this week.

Notation as a Tool of Thought was submitted by Hacker School resident R0ml, who shared the following:

The paper argues for the importance of constructing a more sophisticated form of programming language than any of the ones which have achieved popularity. The paper is easily a hundred years ahead of its time – in that after reading this paper, you look at existing programming languages and realize how crude they are compared to the ones which will have to be developed in order to synthesize the mathematical and linguistic aspects of programming. Mathematica (or the Wolfram Language) is the only programming language in use today which implements these ideas.

Iverson’s paper doesn’t have an abstract, so here’s an excerpt from the introduction:

The importance of nomenclature, notation, and language as tools of thought has long been recognized. In chemistry and in botany, for example, the establishment of systems of nomenclature by Lavoisier and Linnaeus did much to stimulate and to channel later investigation. Concerning language, George Boole in his Laws of Thought asserted “That language is an instrument of human reason, and not merely a medium for the expression of thought, is a truth generally admitted.”

Mathematical notation provides perhaps the best-known and best-developed example of language used consciously as a tool of thought. Recognition of the important role of notation in mathematics is clear from the quotations from mathematicians given in Cajori’s A History of Mathematical Notations. They are well worth reading in full, but the following excerpts suggest the tone:

By relieving the brain of all unnecessary work, a good notation sets it free to concentrate on more advanced problems, and in effect increases the mental power of the race.—A.N. Whitehead

The quantity of meaning compressed into small space by algebraic signs, is another circumstance that facilitates the reasonings we are accustomed to carry on by their aid.—Charles Babbage

Read Along

Read Along is a way for you to participate in Paper of the Week. If you want to take part, all you have to do is read the paper, make something small in response (code or prose), and email us a link to what you made by noon Eastern Time next Monday.

Last week’s paper, The Power of Two Random Choices: A Survey of Techniques and Results, was submitted by Hacker School alum Dan Luu. This week, Dan submitted a Read Along to his own submission about cache eviction and random choice.

You might notice that Dan wrote his submission before last week’s PotW was published. We think that’s great! If you’ve made something in the past that’s based on one of our papers, we’d love to see your submission too.

Happy Thanksgiving (and happy reading)!

David albert 150
Tweet

Paper of the Week: The Power of Two Random Choices: A Survey of Techniques and Results

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is The Power of Two Random Choices: A Survey of Techniques and Results by Michael Mitzenmacher, Andréa W. Richa and Ramesh Sitaraman. It was published in 2001 as part of the book Handbook of Randomized Computing.

This week’s paper was submitted by Hacker School alum Dan Luu, who shared the following:

I like this paper because it’s a really simple idea that’s quite powerful and generally applicable. It’s so simple I can even describe the key intuition in my summary. If you randomly throw n balls into n bins, the maximum number of balls in a single bin is approximately O(log n), with high probability. But if you take two random choices and place the ball in the bin with fewer balls, the maximum drops to O(log log n). The balls into bins model is a natural fit for scheduling / load balancing, hashing, and a number of other common problems. As a result, two random choices often turns out to be really effective despite its simplicity.

It turns out this is also applicable to a wide variety of problems that don’t obviously map to balls / bins, like circuit routing and random graphs (although applying it to things that aren’t “obviously” balls/bins problems isn’t always simple).

This week’s paper doesn’t have an abstract, so here’s the introduction:

To motivate this survey, we begin with a simple problem that demonstrates a powerful fundamental idea. Suppose that n balls are thrown into n bins, with each ball choosing a bin independently and uniformly at random. Then the maximum load, or the largest number of balls in any bin, is approximately log n / log log n with high probability. Now suppose instead that the balls are placed sequentially, and each ball is placed in the least loaded of d ≥ 2 bins chosen independently and uniformly at random. Azar, Broder, Karlin, and Upfal showed that in this case, the maximum load is log log n / log d + Θ(1) with high probability.

The important implication of this result is that even a small amount of choice can lead to drastically different results in load balancing. Indeed, having just two random choices (i.e., d = 2) yields a large reduction in the maximum load over having one choice, while each additional choice beyond two decreases the maximum load by just a constant factor. Over the past several years, there has been a great deal of research investigating this phenomenon. The picture that has emerged from this research is that the power of two choices is not simply an artifact of the simple balls-and-bins model, but a general and robust phenomenon applicable to a wide variety of situations. Indeed, this two-choice paradigm continues to be applied and refined, and new results appear frequently.

Read Along

Read Along is a way for you to participate in Paper of the Week. If you want to take part, all you have to do is read the paper, make something small in response (code or prose), and email us a link to what you made by noon Eastern Time next Monday.

There weren’t any submissions for last week’s paper. We’re excited to get some good ones next week.

Happy reading!

David albert 150
Tweet

Testing from the ground up

Tests are pieces of code that check if your main code works. I write tests to catch bugs when I refactor. I write tests to force myself to think through and handle edge cases. I write tests to show the users of my project that my code does what I say it does.

For this essay, I will describe the code and tests for a tiny web app that draws a blue sky if it’s day time.

Blue sky

And a black sky if it’s night time.

Black sky

I will describe all the code I wrote. The web app. The microscopic testing framework. The tests for the client side code. The mocks that fake layers of the web app and technologies that are not pertinent. The tests that use these mocks. The refactored code that simplifies the mocks by dividing the web app code into different pieces that have single responsibilites.

Along the way, I will talk about many fun things. Temporarily patching libraries written by other people. Writing code that pretends to be the Internet. Making Ajax requests by hand. Writing a little web server. Examining a function to find out how many arguments it expects. Making asynchronous blocks of code run serially so they don’t tread on each other’s toes.

The code

To see the code from this essay in runnable form, go to the testing from the ground up GitHub repository. At each evolution of the code, I will include a link to the corresponding commit.

The web app

This is the HTML that defines the only page in the web app. It has a canvas element that displays the sky. It loads the client side JavaScript, client.js. When the DOM is ready, it calls loadTime(), the one and only function.

<!-- index.html -->

<!doctype html>
<html>
  <head>
    <script src="client.js"></script>
  </head>

  <body onload="loadTime();">
    <canvas id="canvas" width="600" height="100"></canvas>
  </body>
</html>

Below is the client side JavaScript.

loadTime() starts by making an Ajax request to the server to get the current time. This is done in several steps. First, it creates an XMLHttpRequest object. Second, near the bottom of the function, it configures the object to make a GET request to "/time.json". Third, it sends the request. Fourth, when the requested time data arrives at the client, the function bound to request.onload fires.

The bound function grabs the drawing context for the canvas element. It parses "day" or "night" from the JSON returned by the server. If the value is "day", it sets the draw color to blue and draws a rectangle that takes up the whole canvas. If the value is "night", the color is black.

// client.js

var loadTime = function() {
  var request = new XMLHttpRequest();
  request.onload = function(data) {
    var ctx = document.getElementById("canvas").getContext("2d");
    var time = JSON.parse(data.target.responseText).time;
    var skyColor = time === "day" ? "blue" : "black";

    ctx.fillStyle = skyColor;
    ctx.fillRect(0, 0, ctx.canvas.width, ctx.canvas.height);
  };

  request.open("GET", "/time.json", true);
  request.send();
};

Below is the server-side JavaScript. It is run on Node.js. Near the bottom, the code uses the Node HTTP module to create a web server. It specfies that every web request should be handled by requestHandler(). Each time this function is called, Node passes it a request object that has the URL that was requested, and a response object that can be used to send data back to the client.

If the client requested "/", the root, the index.html file is read from disk and its contents are sent back to be displayed in the user’s browser. If "/time.json" was requested, the server looks up the time, creates a piece of JSON that looks something like { "time": "day" } and sends it back to the user’s web browser.

// server.js

var http = require("http");
var fs = require("fs");

var requestHandler = function (request, response) {
  if (request.url === "/") {
    fs.readFile(__dirname + "/index.html", function (err, data) {
      response.writeHead(200, { "Content-Type": "text/html" });
      response.end(data);
    });
  } else if (request.url === "/client.js") {
    fs.readFile(__dirname + "/client.js", function (err, data) {
      response.writeHead(200, { "Content-Type": "application/javascript" });
      response.end(data);
    });
  } else if (request.url === "/time.json") {
    response.writeHead(200, { "Content-Type": "application/json" });

    var hour = new Date().getHours();
    var time = hour > 6 && hour < 20 ? "day" : "night";

    response.end('{ "time": "' + time + '" }');
  }
};

http.createServer(requestHandler).listen(4000);

Here is the code for the basic web app.

A miniscule testing framework

Test code is very simple. It runs application code, and throws an error if the outcome is not as expected. Test code can be structured as functions, where each function tests a specific scenario.

Some of the test code is boiler plate that runs each test in turn, prints the outcome and reports errors. It is nice to abstract this code into a framework to avoid repetition. This is the one I wrote.

// tests/test.js

var test = function() {
  gatherTests(arguments).forEach(function(userTest) {
    userTest();
    process.stdout.write(".");
  });

  console.log();
};

test.isEqual = function(a, b) {
  if (a !== b) {
    throw a + " and " + b + " are not equal";
  }
};

var gatherTests = function(testArgs) {
  return Array.prototype.slice.call(testArgs).filter(function(x, i) {
    return i % 2 === 1;
  });
};

test() could be used to write a test like this:

// example_tests.js

var test = require("./test");
test(
  "should test 1 equals 1", function() {
    test.isEqual(1, 1);
  });

Which, when run, would look like this:

$ node example_tests.js
  .

How does the testing framework run the tests it is given?

test() takes a series of arguments that alternate between string descriptions and test functions. It throws away the ones that are strings, which are merely human-readable window-dressing. It puts the functions into an array. It walks through that array, calling each function and printing a period to indicate that the test passed.

The test functions use test.isEqual() to assert that certain variables have certain values. (A real testing framework would have a more permissive and pragmatic version of isEqual() that returned true in a case like test.isEqual({ love: "you" }, { love: "you" }).) If an assertion fails, an exception is thrown, an error is printed and the tests stop running.

Mocking the server and the Internet

I don’t want to have to run the server when I run the tests. That would be an extra step. It would add statefulness that would need to be reset before each test. It would necessitate network communications between the test and the server.

This is the code I wrote that pretends to be the server deciding what time it is and the Internet relaying that information back to the client. It does this by faking an Ajax request. It replaces the XMLHttpRequest constructor function with a function that returns a home-made Ajax request object. This object swallows the web app’s call to open(). When the web app calls send(), it calls the function that the web app has bound to onload. It passes some fake JSON that makes it seem like it is always day time.

global.XMLHttpRequest = function() {
  this.open = function() {};

  this.send = function() {
    this.onload({
      target: { responseText: '{ "time": "day" }' }
    });
  };
};

Mocking the canvas and the DOM

When the real code runs in a real web browser, it renders the sky to the real canvas in real blue. This is problematic. First, I don’t want to require a browser to test my code. Second, even if I capitulated to that requirement, it would be hard to check if the right thing happened. I would probably need to look at the individual pixels of the canvas drawing context to see if they were bluey.

Instead of walking down that horrid road, I wrote some code that pretends to be the browser DOM and the canvas element. It redefines getElementById() to return a fake canvas. This has a fake getContext() that returns a fake drawing context that has a fake fillRect() and a fake reference to a fake canvas that has a width and height. Instead of drawing, this function checks that the arguments passed to it have the expected values.

global.document = {
  getElementById: function() {
    return {
      getContext: function() {
        return {
          canvas: { width: 300, height: 150 },
          fillRect: function(x, y, w, h) {
            test.isEqual(x, 0);
            test.isEqual(y, 0);
            test.isEqual(w, 300);
            test.isEqual(h, 150);
            test.isEqual(this.fillStyle, "blue");
          }
        };
      }
    };
  }
};

This is the full test.

// tests/client_tests.js

var test = require("./test");
var client = require("../client");

test(
  "should draw blue sky when it is daytime", function() {
    global.XMLHttpRequest = function() {
      this.open = function() {};

      this.send = function() {
        this.onload({
          target: { responseText: '{ "time": "day" }' }
        });
      };
    };

    global.document = {
      getElementById: function() {
        return {
          getContext: function() {
            return {
              canvas: { width: 300, height: 150 },
              fillRect: function(x, y, w, h) {
                test.isEqual(x, 0);
                test.isEqual(y, 0);
                test.isEqual(w, 300);
                test.isEqual(h, 150);
                test.isEqual(this.fillStyle, "blue");
              }
            };
          }
        };
      }
    };

    client.loadTime();
  });

Don’t worry. That code is as bad as this essay gets.

Here is the code that includes the testing framework and the first client side test.

Refactoring the drawing code into its own module

Those mocks are pretty horrid. They make the test very long, which discourages me from writing tests to check for other ways the code might go wrong. The mocks are horrid because the client side code is one big function that communicates with the server, asks what time it is, parses the response and draws in the canvas.

To solve this problem, I refactored the code by pulling the drawing code out into its own module, renderer. This module includes fillBackground(), a function that fills the canvas with the passed color. As a side benefit, the main web app code is now easier to understand and change.

// client.js

var loadTime = function() {
  var request = new XMLHttpRequest();
  request.onload = function(data) {
    var time = JSON.parse(data.target.responseText).time;
    var color = time === "day" ? "blue" : "black";
    renderer.fillBackground(color);
  };

  request.open("GET", "/time.json", true);
  request.send();
};

var renderer = {
  ctx: function() {
    return document.getElementById("canvas").getContext("2d");
  },

  fillBackground: function(color) {
    var ctx = this.ctx();
    ctx.fillStyle = color;
    ctx.fillRect(0, 0, ctx.canvas.width, ctx.canvas.height);
  }
};

This lets me replace the complex document mock with a short renderer.ctx() mock. The test becomes shorter, simpler and less brittle.

// tests/client_tests.js

var test = require("./test");
var client = require("../client");

test(
  "should draw blue sky in canvas when it is daytime", function() {
    global.XMLHttpRequest = function() {
      this.open = function() {};

      this.send = function() {
        this.onload({
          target: { responseText: '{ "time": "day" }' }
        });
      };
    };

    client.renderer.ctx = function() {
      return {
        canvas: { width: 300, height: 150 },
        fillRect: function(x, y, w, h) {
          test.isEqual(x, 0);
          test.isEqual(y, 0);
          test.isEqual(w, 300);
          test.isEqual(h, 150);
          test.isEqual(this.fillStyle, "blue");
        }
      }
    };

    client.loadTime();
  });

Here is the code for the modularised renderer and resulting simplified client side test.

Refactoring the network code and business logic

I wrote three more functions that split the code into pieces, each with a different responsiblity. This will let me write better tests.

This is get(). It makes an Ajax request to the passed URL. The Ajax request object calls the passed callback with the response.

var get = function(url, callback) {
  var request = new XMLHttpRequest();
  request.onload = callback;
  request.open("GET", url, true);
  request.send();
};

This is getTime(). It uses get() to make an Ajax request to "/time.json" and parses "day" or "night" from the response.

var getTime = function(callback) {
  get("/time.json", function(data) {
    callback(JSON.parse(data.target.responseText).time);
  });
};

This is displayTime(). It takes a string with the value "day" or "night" and draws either a blue sky or a black sky.

var displayTime = function(time) {
  var color = time === "day" ? "blue" : "black";
  renderer.fillBackground(color);
};

I changed the body tag in the HTML page. It now calls getTime(), passing displayTime() as the callback.

<body onload="getTime(displayTime);">
  <canvas id="canvas" width="600" height="100"></canvas>
</body>

Having more modular code means I can mock parts of the web app API, rather than mocking browser code written by third parties. This makes each test for a specific piece of functionality more succinct, which makes it easier to write more extensive tests.

The first test checks that getTime() correctly parses JSON sent by the server. The second test checks that a call to fillBackground() draws a rectangle at the correct position and size. The third test checks that displayTime() draws a rectangle of the correct color for the time of day.

// tests/client_tests.js

var test = require("./test");
var client = require("../client");

test(
  "should parse time from server", function() {
    client.get = function(_, callback) {
      callback({ target: { responseText: '{ "time": "day" }' }});
    };

    client.getTime(function(time) { test.isEqual(time, "day"); });
  },

  "should draw rect of size and color when fillBackground() called", function() {
    client.renderer.ctx = function() {
      return {
        canvas: { width: 300, height: 150 },
        fillRect: function(x, y, w, h) {
          test.isEqual(x, 0);
          test.isEqual(y, 0);
          test.isEqual(w, 300);
          test.isEqual(h, 150);
        }
      }
    };

    client.renderer.fillBackground("blue");
  },

  "should draw blue sky when it is daytime", function() {
    client.renderer.fillBackground = function(color) {
      test.isEqual(color, "blue");
    };

    client.displayTime("day");
  });

Here is the code for the more modular client code and more extensive tests.

Testing the server side code

Asynchronous tests

Some of the responsibilities of a web server require asynchronous operations. If a browser sends a request to the web app for the URL "/", it gets back the contents of the index.html file. To make this happen, the file needs to be read from disk asynchronously and the contents sent to the client once they have been read. Similarly, if the browser requests the URL "/client.js", it gets back the contents of the client.js file.

These are the naïve tests I wrote to check that both cases are handled correctly.

// tests/server_tests.js

var test = require("./test");
var requestHandler = require("../server").requestHandler;
var http = require("http");

test(
  "should send index.html when / requested", function() {
    http.ServerResponse.prototype.end = function(data) {
      test.isEqual(data.toString().substr(0, 9), "<!doctype");
    };

    requestHandler({ url: "/" }, new http.ServerResponse({}));
  },

  "should send client.js when /client.js requested", function() {
    http.ServerResponse.prototype.end = function(data) {
      test.isEqual(data.toString().substr(0, 10), ";(function");
    };

    requestHandler({ url: "/client.js" }, new http.ServerResponse({}));
  });

But here is the output when I run these tests:

$ node server_tests.js
  ..

  test.js:12
    throw a + " and " + b + " are not equal";
                          ^
  <!doctype  and ;(function are not equal

An exception is thrown, yet there are two periods indicating two passed tests. Something is wrong.

In the first test, response.end() is mocked with a function that checks that
"<!doctype" is sent to the user. Next, requestHandler() is called, requesting the URL "/". requestHandler() starts reading index.html from disk. While the file is being read, the test framework presses on with its work. Uh oh. That is, the framework prints a period and starts the second test, though the response.end() mock has not asserted the value of the response. response.end() is re-mocked with a function that checks that ";(function" is sent to the user. Double uh oh. requestHandler() is called by the second test. It requests the URL "/client.js". requestHandler() starts reading client.js from disk. The framework prints another premature period.

At some point later, index.html is read from disk. This means that the callback in requestHandler() is called and it calls response.end() with the contents of index.html. Unfortunately, by this time, response.end() has been mocked with a function expecting ";(function". The assertion fails.

This problem can be solved by running tests serially. A test is run. It signals it has finished. The next test is run.

This may seem pedantic. Shouldn’t tests that are testing asynchronous behaviour be able to cope with with the dangers of asynchrony? Well, yes and no. They should certainly test the asynchronous behaviours of requestHandler(). But they should not have to cope with other tests messing about with their execution environment part way through their execution.

(It would be possible to go further and make the tests completely functionally pure. This could be done in a fundamentalist way: the test framework resets the execution context before each test. Or it could be done in a pragmatic way: each test undoes the changes it made to the execution environment. Both ways are outside the scope of this essay.)

I rewrote the testing framework to run asynchronous tests serially. Each asynchronous test binds a done callback parameter. It calls this when it has made all its assertions. The testing framework uses the execution of this callback as a signal to run the next test. Here are the rewritten tests.

// tests/server_tests.js

var test = require("./test");
var requestHandler = require("../server").requestHandler;
var http = require("http");

test(
  "should send index.html when / requested", function(done) {
    http.ServerResponse.prototype.end = function(data) {
      test.isEqual(data.toString().substr(0, 9), "<!doctype");
      done();
    };

    requestHandler({ url: "/" }, new http.ServerResponse({}));
  },

  "should send client.js when /client.js requested", function(done) {
    http.ServerResponse.prototype.end = function(data) {
      test.isEqual(data.toString().substr(0, 10), ";(function");
      done();
    };

    requestHandler({ url: "/client.js" }, new http.ServerResponse({}));
  });

A miniscule asynchronous testing framework

Below is the code for the asynchronous testing framework.

Look at runTests(). It takes userTests, an array that contains the test functions to be run. If that array is empty, the tests are complete and the program exits. If it is not empty, it looks at the length attribute of the next test function. If the attribute has the value 1, the test expects one argument: a done() callback. It runs testAsync(), passing the test function and a callback that prints a period and recurses on runTests() with the remaining test functions.

testAsync() creates a timeout that will fire in one second. It runs the test function, passing a done() callback for the test to run when it is complete. If the callback gets run, the timeout is cleared and testDone() is called to indicate that the next test can run. If the done() callback is never run by the test function, something went wrong. The timeout will fire and throw an exception, and the program will exit.

If the length attribute of the test function has the value 0, the function is run with testSync(). This is the same as testAsync(), except there is no timeout and the testDone() callback is called as soon as the test function has completed.

// tests/test.js

var test = function() {
  runTests(gatherTests(arguments));
};

var runTests = function(userTests) {
  if (userTests.length === 0) {
    console.log();
    process.exit();
  } else {
    var testDone = function() {
      process.stdout.write(".");
      runTests(userTests.slice(1));
    };

    if (userTests[0].length === 1) {
      testAsync(userTests[0], testDone);
    } else {
      testSync(userTests[0], testDone);
    }
  }
};

var testSync = function(userTest, testDone) {
  userTest();
  testDone();
};

var testAsync = function(userTest, testDone) {
  var timeout = setTimeout(function() {
    throw "Failed: done() never called in async test.";
  }, 1000);

  userTest(function() {
    clearTimeout(timeout);
    testDone();
  });
};

test.isEqual = function(a, b) {
  if (a !== b) {
    throw a + " and " + b + " are not equal";
  }
};

var gatherTests = function(testArgs) {
  return Array.prototype.slice.call(testArgs).filter(function(x, i) {
    return i % 2 === 1;
  });
};

Here is the code for the asynchronous testing framework and the new server tests.

An exercise

Now that it is possible to write asynchronous tests, I can write the tests for the server. Or, rather: you can.

If you are not sure where to start, try refactoring the server so the code is more modular. Write a function that sends the passed string with the passed Content-Type to the client. Write a function that reads a static file from disk and responds with the contents. Write a function that converts the current date into "day" or "night". Write a function that takes a request for a certain URL and sends the right response.

Once your refactor is complete, or maybe while it is in progress, you can write your tests.

Summary

I wrote a simple web app. I wrote tests for it and discovered that I could mock the pieces I didn’t want to run when I ran the tests. I discovered that scary things like Ajax and the canvas API really aren’t so scary when I wrote skeletal versions of them. I realised that the mocks I had written were quite verbose. I refactored the web app code to make it more modular. This made the code better and easier to change in the future. It meant I could mock the interfaces I had written, rather than those invented by other people. This meant the mocks became simpler or unnecessary. This made it easier to write tests, so I could write more of them and test the web app more extensively.

I wrote two tests for the server. I discovered that the test framework ran them in parallel, which meant they interfered with each other. I rewrote the test framework to run tests serially. I modified the tests to signal when they were finished. I handed over the rest of the job to you.

Mary rose cook 150
Tweet

Paper of the Week: Worlds: Controlling the Scope of Side Effects

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is Worlds: Controlling the Scope of Side Effects by Alessandro Warth, Yoshiki Ohshima, Ted Kaehler and Alan Kay. It was presented at ECOOP 2011 and is based on part of Warth’s Ph.D. thesis Experiments with Programming Languages. When this paper was published, all four authors were researchers at Viewpoints Research Institute.

Worlds was submitted by Hacker School resident David Nolen, who shared the following:

Alan Kay’s VPRI group published this fascinating paper on a general pattern for controlling the scope of side effects. The formulation is object oriented in flavor but the observations and results are significant and may be arrived at by other means. Specifically, in the closing “Related” section they note that Worlds deliver the same benefits made possible by persistent data structures. This is one of the key papers that inspired my work around Om.

Compare the undo implementation I described for an Om based TodoMVC and the undo implementation described in section 3.2. They are surprisingly similar (I had not actually read the source in this section when devising my example). In section 3.3 they describe extension methods - this is available in ClojureScript via specify. You can think of specify as an instance level type class extension that returns a new extended (equal) value. Internally Om uses this feature all over the place. In ClojureScript extension methods are very high performance - deep cloning with persistent data structures is O(1).

I think in the long run this paper will likely be one of the seminal references for anyone trying to reconcile the powerful modularity of stateful Object Oriented programming with the stunning properties derived from value oriented functional programming - particularly when attempting to apply these strategies to the computer programming domains that still exhibit staggering complexity at scale (this is what VPRI is interested in) - namely UI programming.

Here’s the abstract from the paper:

The state of an imperative program—e.g., the values stored in global and local variables, arrays, and objects' instance variables—changes as its statements are executed. These changes, or side effects, are visible globally: when one part of the program modifies an object, every other part that holds a reference to the same object (either directly or indirectly) is also affected. This paper introduces worlds, a language construct that reifies the notion of program state and enables programmers to control the scope of side effects. We investigate this idea by extending both JavaScript and Squeak Smalltalk with support for worlds, provide examples of some of the interesting idioms this construct makes possible, and formalize the semantics of property/field lookup in the presence of worlds. We also describe an efficient implementation strategy (used in our Squeak-based prototype), and illustrate the practical benefits of worlds with two case studies.

Read Along

Read Along is a way for you to participate in Paper of the Week. If you want to take part, all you have to do is read the paper, make something small in response (code or prose), and email us a link to what you made by noon Eastern Time next Monday.

This week Brian Cobb wrote a reflection about the the Bayou paper with an explanation of the Write Log and the Undo Log. Thanks Brian!

Happy reading!

David albert 150
Tweet

What people do at Hacker School (Nov 2014)

This is a follow-up to my earlier post, What people do at Hacker School. As before, the goal is to answer the deceptively simple question of what people do at Hacker School. Because Hacker School projects are as diverse as Hacker Schoolers themselves, the best way to answer this by example.

So here’s another small sampling of things people have done at Hacker School in the last few weeks:

  • Create language for generating minimalist art. Sarah is building LeWittScript, a conceptualist art and code project for generating art in the style of the American artist Sol LeWitt.

  • Build software to broadcast terminal sessions. Jesse wrote a reimplementation of Termcast in Python to let people share their terminal sessions over SSH and the web.

  • Contribute back to Community (in ClojureScript). Madhu and Suren are adding a search feature to Community, the open source forum software we replaced Google Groups with internally this year.

  • Write a Twitter-based file system. Adrien wrote a FUSE-based file system on top of Twitter so you can cat, ls and rm your followers.

  • Pair program and then write a blog post – together. Susan and Stephanie added a feature to bpython, and then pair blogged the experience.

  • Update a kernel module to fix your keyboard. Chase’s keyboard wasn’t working how he wanted it to, so he read the Linux source, wrote a patch, and blogged about the process.

If you’d like to spend three months becoming a better programmer and working on projects like these, you should apply to Hacker School. Don’t let any of these projects intimidate you: Most people report that Hacker School enabled them to tackle things they previously thought they couldn’t.

Nick bergson shilcock 150
Tweet

Paper of the Week: Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System by Douglas B. Terry, Marvin M. Theimer, Karin Petersen, Alan J. Demers, Mike J. Spreitzer and Carl H. Hauser. It was written at Xerox’s Palo Alto Research Center and presented at SOSP ‘95, the ACM Symposium on Operating Systems Principles.

This week’s paper was submitted by Hacker Schooler Maggie Zhou, who shared the following:

Designing a storage system in the face of bad networks and update conflicts, are still problems today (see: the network is (not) reliable). Bayou presents an early example of designing in the face of network partition and conflicting, concurrent updates. It was the first systems paper I got through and really made me want to read more.

Here’s the abstract from the paper:

Bayou is a replicated, weakly consistent storage system designed for a mobile computing environment that includes portable machines with less than ideal network connectivity. To maximize availability, users can read and write any accessible replica. Bayou’s design has focused on supporting application-specific mechanisms to detect and resolve the update conflicts that naturally arise in such a system, ensuring that replicas move towards eventual consistency, and defining a protocol by which the resolution of update conflicts stabilizes. It includes novel methods for conflict detection, called dependency checks, and per-write conflict resolution based on client-provided merge procedures. To guarantee eventual consistency, Bayou servers must be able to rollback the effects of previously executed writes and redo them according to a global serialization order. Furthermore, Bayou permits clients to observe the results of all writes received by a server, including tentative writes whose conflicts have not been ultimately resolved. This paper presents the motivation for and design of these mechanisms and describes the experiences gained with an initial implementation of the system.

Read Along

Read Along is a way for you to participate in Paper of the Week. If you want to take part, all you have to do is read the paper, make something small in response (code or prose), and email us a link to what you made by noon Eastern Time next Monday.

Last week’s paper was Out of the Tar Pit. Here are the Read Along submissions:

  • Igor Bondarenko wrote a reflection on how the paper’s proposed Functional Relational Programming system relates to some current ideas in UI programming.
  • Dan Luu, a Hacker School alum, takes issue with the paper’s claim that informal reasoning is more important than testing.

We also had one more Read Along submission for the Chubby paper from two weeks ago:

Happy reading!

David albert 150
Tweet

Jean Yang and Matt Might are Hacker School Residents

We’re excited to announce that Jean Yang and Matt Might will be in residence at Hacker School during our upcoming Winter 1 batch.

Jean is an MIT PhD student studying programming languages. She created Jeeves and Haskell Ryan Gosling and also worked on Verve, an operating system verified end-to-end for type safety.

Matt is an Associate Professor at the School of Computing at the University of Utah who researches data structures, meta-programming and parsing with derivatives, and teaches a course on compilers.

Jean will be in residence at Hacker School from November 17th – November 20th, and Matt will be in residence from December 1st – December 4th.

If you’re as excited to work with Matt and Jean as we are, apply to our Winter 1 batch!

We’ve been lucky to have some amazing residents this summer and fall. Crista Lopes, José Valim, Leigh Honeywell, Peter Seibel, Jessica McKellar and Paul Tagliamonte all joined us earlier this year, and we’re grateful to have spent time programming with and learning from them all.

Rachel vincent 150
Tweet

Paper of the Week: Out of the Tar Pit

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is Out of the Tar Pit by Ben Moseley and Peter Marks, which was presented at SPA2006.

Out of the Tar Pit was submitted by Hacker Schooler Pablo Torres, who shared the following:

In Out of the Tar Pit, Moseley and Marks argue that the biggest source of complexity in programs is mutable state, because keeping track of all possible combinations of bits is unmanageable. They introduce the notions of essential and incidental complexity, the former referring to details that are inherent to a problem and the latter to those complications that arise due to a particular solution one might attempt - for example, issues added by the programming language in use.

They continue by comparing Object-Oriented Programming’s approach of breaking state up into small, manageable pieces and providing controlled mutations with Functional Programming’s solution of avoiding mutations altogether; they also spend some time analyzing Logic Programming. They favor the simplicity of FP, even though they admit that it takes discipline to achieve.

Finally, they offer recommendations on how to build systems that are easy to reason about by using the tools provided by Functional Programming and relational database modelling.

Here’s the abstract from the paper:

Complexity is the single major difficulty in the successful development of large-scale software systems. Following Brooks we distinguish accidental from essential difficulty, but disagree with his premise that most complexity remaining in contemporary systems is essential. We identify common causes of complexity and discuss general approaches which can be taken to eliminate them where they are accidental in nature. To make things more concrete we then give an outline for a potential complexity-minimizing approach based on functional programming and Codd’s relational model of data.

Read Along

Read Along is a way for you to participate in Paper of the Week. If you want to take part, all you have to do is read the paper, make something small in response (code or prose), and email us a link of what you make by noon Eastern Time next Monday.

Last week’s paper was The Chubby Lock Service for Loosely-Coupled Distributed Systems. Here are the Read Along submissions:

Happy reading!

David albert 150
Tweet

Hacker School Profile: Stephanie Samson

This is the first of what may become a series of posts highlighting Hacker Schoolers. The goal is to share more about the many paths people take before, during, and after Hacker School.

Our first profile post is on Stephanie Samson, who’s a current Hacker Schooler in the Fall 1, 2014 batch. Steph kindly agreed to take 15 minutes out of her day to talk with me about what she did before Hacker School, and what she’s working on now.

Steph

How did you start programming?
I really started right around high school. I was working with the Santa Fe Institute mentoring middle school students about science and math, and that’s where I got my first exposure to programming. But I didn’t dive deeply into it until December of last year, when I went to a hackathon in Australia. I joined a team as a designer, but I started learning Ruby.

What did you do next?
I just spent a month focused on teaching myself Ruby, and then I decided to apply for some jobs. I never thought I’d get any offers, but I did. I worked at Gramercy Studios until August, when I came to Hacker School.

What have you worked on at Hacker School so far?
I’ve worked on a Scheme interpreter, a translation bot, and an API wrapper for Google Translate, all in Python. I’ve also been learning some more CS fundamentals, like recursion and linked lists, which I’ve found to be very helpful since prior to Hacker School I didn’t have any experience with more “computer science-y” stuff.

When working on the Scheme interpreter I decided to try test-driven development. I would first write tests for the output that my interpreter should have for different inputs, and then make them pass. I was just getting started with TDD, and I was stressed about whether or not I was approaching it correctly, so I sat down with Zach, Tom and Jesse [facilitators and a fellow Hacker Schooler] and they helped me get on the right track.

Now I’m trying to encourage Susan [a fellow Hacker Schooler], who’s also writing an interpreter, to use TDD.

How come you chose to write an API wrapper?
I wanted to make a translation bot for Zulip [Hacker School’s internal chat system]. And so I looked for an API wrapper for Google or Bing Translate, but I couldn’t find a good one, so I decided to make my own. It was a really interesting project for me because I’ve never made anything like it. I had to first understand what makes a good API wrapper, and so I had to read other people’s code for other API wrappers. The big lesson that I learned is to read other people’s code; I found this to be more helpful than even reading people’s blog posts about their projects. This is something I’ve definitely gotten better at Hacker School – reading other people’s code.

How have you found pair programming with other people?
I’ve really enjoyed pairing with people. It’s great to get to work with people who are genuinely curious, who want to become better programmers, and who aren’t judgemental. It’s great to be able to ask questions and not get a “well, actually” or have someone feign surprise. It’s also really helped me with concentration, and just focusing on one thing at a time. I think that’s more important than multitasking.

What are you planning to work on next?
I’m going to learn more about functional programming using Elm. There are several people here working on Elm, and I find the community really helps you learn faster. I made much faster progress with Python thanks to the people here, and I think the same will be true with Elm.

Thanks to Stephanie Samson for being a wonderful part of the Hacker School community, and for taking the time to chat for this interview. If you’d like to join a supportive, energizing, and intellectually curious community, there’s still time to apply for our Winter 1 batch which begins on November 17th.

Nick bergson shilcock 150
Tweet

Paper of the Week: The Chubby Lock Service for Loosely-Coupled Distributed Systems

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is The Chubby Lock Service for Loosely-Coupled Distributed Systems. It was written by Mike Burrows at Google and was presented at OSDI'06, USENIX’s Symposium on Operating Systems Design and Implementation.

This week’s paper was submitted by Hacker Schooler Leah Hanson, who said the following:

Chubby is a distributed lock service; it does a lot of the hard parts of building distributed systems and provides its users with a familiar interface (writing files, taking a lock, file permissions). The paper describes it, focusing on the API rather than the implementation details; it is written in a very readable style. There are amusing stories about other Google engineers (their users) using the API incorrectly; in response, they either fix the API or the implementation so that it’s no longer a problem.

Chubby provides locks and files. Distributed systems usually have one master (in a database, the master approves all writes before they’re real); when the master dies, a new machine needs to be elected master. Without Chubby, this is a hard problem and implementations are error prone. With Chubby (which depends on Paxos, the gold standard for this), a machine will try to grab a lock & write it’s name in a file when it wants to elect itself master; if it succeeds, it’s the new master and the other machines will believe it once they read it’s name in the file. Chubby provides an simple interface, but still gives you the correctness of using Paxos (which is very complex to use).

You have to work at Google if you want to see Chubby’s source code, but ZooKeeper is an opensource alternative.

Here’s the abstract from the paper:

We describe our experiences with the Chubby lock service, which is intended to provide coarse-grained locking as well as reliable (though low-volume) storage for a loosely-coupled distributed system. Chubby provides an interface much like a distributed file system with advisory locks, but the design emphasis is on availability and reliability, as opposed to high performance. Many instances of the service have been used for over a year, with several of them each handling a few tens of thousands of clients concurrently. The paper describes the initial design and expected use, compares it with actual use, and explains how the design had to be modified to accommodate the differences.

Read Along

Read Along is a way for you to participate in Paper of the Week. If you want to take part, all you have to do is read the paper, make something small in response (code or prose), and email us a link of what you make by noon Eastern Time next Monday.

Last week’s paper was The Power of Interoperability: Why Objects Are Inevitable. Here are the Read Along submissions:

Happy reading!

David albert 150
Tweet

Paper of the Week: The Power of Interoperability: Why Objects Are Inevitable

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is The Power of Interoperability: Why Objects Are Inevitable by Jonathan Aldrich an Associate Professor at Carnegie Mellon University. It was published in Onward! Essays, 2013

I submitted this paper because it’s a good companion to last week’s. Aldrich argues that the essential thing about objects is that they can provide services that operate at a higher level than just data abstraction. I don’t think this paper invalidates last week’s. It’s perspective is different and complimentary.

This paper also draws on The Early History Of Smalltalk by Alan Kay, which could easily be called The Early History of Object-Oriented Programming. Aldrich’s perspective on objects seems much closer to Kay’s original ideas than Cook’s does.

When I found Aldrich’s paper on the front page of Hacker News last week, I had just finished reading Kay’s. If you have time, Kay’s paper is a nice companion to this one.

At the suggestion of Hacker Schooler Leah Hanson, we’re going to be posting the abstract from each paper to whet your appetite. Here’s this week’s:

Three years ago in this venue, Cook argued that in their essence, objects are what Reynolds called procedural data structures. His observation raises a natural question: if procedural data structures are the essence of objects, has this contributed to the empirical success of objects, and if so, how?

This essay attempts to answer that question. After reviewing Cook’s definition, I propose the term service abstractions to capture the essential nature of objects. This terminology emphasizes, following Kay, that objects are not primarily about representing and manipulating data, but are more about providing services in support of higher-level goals. Using examples taken from object-oriented frameworks, I illustrate the unique design leverage that service abstractions provide: the ability to define abstractions that can be extended, and whose extensions are interoperable in a first-class way. The essay argues that the form of interoperable extension supported by service abstractions is essential to modern software: many modern frameworks and ecosystems could not have been built without service abstractions. In this sense, the success of objects was not a coincidence: it was an inevitable consequence of their service abstraction nature.

Read Along

Read Along is a way for you to participate in Paper of the Week. If you want to take part, all you have to do is read the paper, make something small in response (code or prose), and email us a link of what you make by noon Eastern Time next Monday.

We’d we’d love to get your feedback on what changes we could make to Paper of the Week to make it even more likely that you’ll submit a Read Along. If there are things we could change to make PotW more welcoming, please let us know.

Happy reading!

David albert 150
Tweet

An unusual goodbye

Allison Kaptur, a long-time Hacker School facilitator and dear friend, has decided to take the next step in her career and join the team at Dropbox. Allison’s contributions to Hacker School are innumerable, and we’re sad to see her go, though we are happy that she will be joining many Hacker School friends at Dropbox.

Like many things at Hacker School, Allison’s departure is atypical. Allison told me in our weekly one-on-one a couple of months ago that she was ready to move to a pure programming role at a software company, and so continuing to be a Hacker School facilitator was no longer the best way to pursue her personal and professional goals. I was sad, but quickly understood that this is indeed the right path for her after hearing her reasoning.

Being transparent here turned out to have a lot of benefits, both for Hacker School and for Allison.

For Hacker School, it meant time to plan, and an opportunity to help Allison choose a minimally disruptive time to depart. As a small company, it’s especially tough to lose a key employee, and having a significant heads up is helpful. It also gave Hacker School the opportunity to be involved in Allison’s job search (our business model is, after all, recruiting).

For Allison, it meant she didn’t have to lie to or be evasive with us or her colleagues, or worry about finding time to interview or take surreptitious trips to California. It also meant she could use us for references and referrals for potential employers, and draw upon Hacker School’s resources and community for interview prep and negotiation advice.

Many companies and people are unnecessarily closed off about their future plans, opting to pretend everyone’s jobs will be their last. But that’s rarely true nor productive. My cofounders and I have had similarly candid conversations with other employees when they’ve been approached by other companies or were considering medical school. Each time, we have tried to be frank about our goals and to hear and understand our employees', and to genuinely try to find what’s best for Hacker School and our employees. I don’t think this would work in all contexts or for all people, but it has worked well for us.

Thank you, Allison, for helping build Hacker School these past years, and for continuing to contribute, even as you move on. We wish you the best of luck, and can’t wait to see what you accomplish next.

Nick bergson shilcock 150
Tweet

Paper of the Week: On Understanding Data Abstraction, Revisited

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is On Understanding Data Abstraction, Revisited by William Cook, an Associate Professor of Computer Science at The University of Texas at Austin. It was originally presented at OOPSLA ‘09. Among his many accomplishments, Cook was one of the original designers of AppleScript. He wrote a history of the language, which you can find on his website. He’s also currently fighting cancer, so if you enjoy his paper and have a bit of money to spare, you can donate to his treatment.

This week, I decided to submit a paper myself. Alan O'Donnell, one of our facilitators, told me about this paper last year and it has had a profound effect on how I think about programming. The paper compares two forms of data abstraction, objects – the kind you find in Smalltalk, Java, Ruby, etc. – and abstract data types (ADTs), which include C’s built-in primitive types and the algebraic data types that are common to the ML family of languages. Cook’s thesis is that while people often conflate the two, objects and ADTs are two distinct forms of data abstraction, which have different properties and use-cases.

Reading this paper felt like lifting a mental fog. Every programming problem that I approached suddenly had two possible solutions instead of one: I could solve the problem with objects, or I could solve it with ADTs. I mostly program in Ruby and JavaScript, which makes it more likely that I’ll solve my problems with objects (both languages have stronger support for objects than ADTs), but even the knowledge that there’s another option has been a useful tool for me in my everyday programming.

Read Along

Read Along is a way for you to participate in Paper of the Week. If you want to take part, all you have to do is read the paper, make something small in response (code or prose), email us a link of what you make by noon Eastern Time next Monday, and we’ll link to it from our blog.

Hacker School facilitator Tom Ballinger is back with another Read Along submission. In it, he works through parts of last week’s paper in Python while reflecting on his own reading and learning styles.

Happy reading!

David albert 150
Tweet

Spring 2, 2015 applications are open

Applications are now open for the Spring 2, 2015 batch of Hacker School, which runs from March 30th through July 2nd. There are also plenty of spots left in the Winter 1, Winter 2, and Spring 1 batches. You can see the dates for these batches in our FAQ. The application deadline for Spring 2 is March 9th, 2015, but admissions are rolling, so the earlier you apply, the better.

Hacker School is free for everyone, but we know that living in NYC can be expensive. We offer need based grants to cover living expenses for people who identify as women (cis or trans), black, Latino/a, Native American, and/or Pacific Islander.

If you’re interested in applying to Hacker School, we strongly encourage you to read our about page, FAQ and User’s Manual.

David albert 150
Tweet

Paper of the Week: Error Detecting and Error Correcting Codes

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week’s paper is Error Detecting and Error Correcting Codes by Richard Hamming. It was published in the April 1950 issue of the Bell System Technical Journal. Hamming worked at Bell Labs from 1946 to 1976. During those years, the transistor, information theory, modern cryptography, the solar cell, the laser, UNIX, C, and the error correcting codes described in this paper were all invented at Bell Labs.

Hacker School alum Dan Luu submitted this paper and said the following:

Despite being more than six decades old, Hamming codes are probably the most commonly used error-correcting codes (ECC) today, and I’ve probably used/implemented them more often than all other types of ECC combined. They’re one of the first things you should consider if you’re sending/storing data in a lossy medium and you need ECC that’s simple/fast, or if you’re operating on small chunks of data.

Also, I believe this is the second paper written on error correcting codes, and that it’s often considered to be the seminal work in the field. Because the field was so young at the time, the paper assumes no background, which makes it eminently readable.

Read Along

Read Along is a way for you to participate in Paper of the Week. If you want to take part, all you have to do is read the paper, make something small in response (code or prose), email us a link of what you make by noon Eastern Time next Monday, and we’ll link to it from our blog.

Nobody submitted a Read Along this week, so there’s nothing to post here. If you are reading along with us, please don’t be shy! Even a one paragraph Gist with your unedited thoughts would be fun for other people to read.

Happy reading!

David albert 150
Tweet

Building a better and more diverse community

The short: We now have need-based living expense grants for blacks and Latino/as, as well as people from other groups traditionally underrepresented in programming. Etsy, Juniper, Perka, Stripe, Betaworks, and Fog Creek have partnered with us to fund the grants, and help make the demographics of Hacker School better reflect those of the US. Hacker School remains free for everyone.

Some history

In early 2012, we partnered with Etsy and began offering need-based grants to women who couldn’t afford to pay for living expenses during Hacker School. This was part of a larger effort to make Hacker School more gender-balanced, and it has worked extremely well.

Over the past two and a half years our community has gone from being under 5% women to being 35%, and from being a few dozen people to nearly 450.

Simultaneously, and in many ways because of this, Hacker School has become a better experience and community along almost every dimension.

Why diversity is important to Hacker School

There are many reasons why diversity is valuable, but there are two reasons why it is especially important to Hacker School as an organization.

The first is that diversity helps to reduces the harmful effects of stereotype threat. Put another way, we focus on diversity so Hacker Schoolers don’t have to. We want you to be able to focus on becoming a better programmer, not being the only person like you in the room. The more diverse Hacker School is, the easier it is for a greater range of people to do that.

Second, a large part of the value of Hacker School is what Hacker Schoolers learn from each other. Hacker School relies on a diverse range of experiences and perspectives; if everyone were the same, no one would have anything to learn from each other! Our self-directed and peer-driven educational model is in this way very different from traditional, one-size-fits-all approaches to education.1

Given this, it’s unsurprising that Hacker School has gotten better as our community has grown to include more women, trans people, genderqueer people, older people, younger people, parents, and people from a greater range of racial, ethnic, and socioeconomic backgrounds.

A long way to go

But while we’re proud of the progress we’ve made, we’re still far from where we want or need to be: Hacker School is still not 50% women and, like the programming world at large, it is still overwhelmingly white and Asian. And while we strive to eliminate it, our community isn’t free of sexism or racism.

Today, we’re excited to take the next step in building a stronger and more diverse community, and announce the largest expansion of our grants program to date. We’ve secured grant funding for our next full year of batches, and are expanding our grants program to support people from traditionally underrepresented racial and ethnic groups in programming, in addition to women.

This is only possible because of the outstanding support and commitment of our partner companies. Etsy has committed over $200,000 in grants for the coming year, and Juniper has pledged an additional $100,000 for grants. Etsy and Juniper have been the two greatest champions and supporters of female programmers at Hacker School, and we are honored to have their continued and expanded support.

We’re also thrilled to announce four new grant sponsors: Betaworks, Fog Creek, Perka, and Stripe. These companies are deeply committed to supporting and increasing diversity in programming and are all sponsoring multiple grants over the next year. We couldn’t be happier to be working with them.

We applaud these companies for not just talking about the importance of supporting diversity, but for stepping up and doing one of the most effective things we’re aware of: Giving money directly to those who need it, with no strings attached.

The details

Our grants are now available to people who identify as women, black, Latino/a, Native American, and Pacific Islander. Since Hacker School charges no tuition, the grants are intended to be used for living expenses during your three months of Hacker School: You can use your grant for housing, food, childcare, or anything else you need during your time here. Each grant is for between $500 and $7,000, depending on the need of the recipient.

Hacker School is in the unusual position to be able to support both relatively new and very experienced programmers from a wide range of backgrounds. There are other organizations focusing purely on getting more people from underrepresented groups into programming and we think that’s an important part of the puzzle. But we also think it’s vitally important to additionally support people from underrepresented groups who are already working as programmers and who struggle with the pervasive discrimination in our industry. The lack of black and female programmers is partially because of how few initially enter the field, but also because of how many are driven out by racism, sexism and unsupportive environments.

So, whether you are entirely self-taught or you have a CS degree, we want you at Hacker School. And regardless of if programming is just something you do for fun or if programming is something you’ve been doing professionally for years, we want you at Hacker School. People with everything from three months to 20 years of programming experience have told us they found Hacker School to be a life-changing experience.

Not lowering the bar

As we said when we announced our grants for women two and a half years ago, we are in no way lowering our admissions bar for anyone. We hold everyone who applies to an equal standard, and no man has ever been denied admission to Hacker School because a woman was admitted. We accept every person who applies who we believe we’re a good fit for and who would benefit from and contribute to Hacker School.

We are always eager to reduce bias in our admissions process, and so this year we began automatically generating pseudonyms for applicants. Instead of seeing names like “José Smith” and “Kimberly Lin” we now see names like “Croissant Wave” and “Representative Mint” when doing our initial application review. We conduct Skype interviews and applications still contain personally identifying information like GitHub links, so the pseudonyms are only intended to eliminate unconscious bias in the first minute or two of our application review.

The data from our past two years of batches has been remarkably consistent: The percent of women in a given batch is statistically indistinguishable from the percent of women who applied to that batch. If 35% of applicants for a batch are women then the batch will be approximately 35% women. We’ve found the same to be true at every step of our admissions process: Men and women are invited to interview, advanced to a second interview, and admitted at the same rates. We believe this provides strong evidence that we’ve succeeded in our commitment to holding women and men to the same admissions standards.2

This also suggests that one of the bests paths to a vibrant and diverse community is outreach: Our community has grown larger and stronger not by turning people away, but by welcoming people from a broader range of backgrounds to apply in the first place.

If you want to spend three months programming with friendly peers with a wide array of experience levels and perspectives, you should apply to Hacker School. And even if Hacker School isn’t what you’re looking for, please help us spread the word.

Current grant sponsors

  • Etsy w110
  • Juniper networks w110

  • Betaworks w110
  • Fog creek w110
  • Perka w110
  • Stripe w110

Please contact us if you’re interested in supporting diversity and sponsoring grants at Hacker School.

Update 10/1/14: After making this announcement, we were told that “Hispanic” is a word few people identify with and many find hurtful. We’re sorry that we weren’t as informed as we could have been and we’re grateful to everyone who let us know how we could do better. We’ve removed “Hispanic” from this blog post and from the rest of our site. We’ve also replaced “non-white Latino/a” with “Latino/a” because it’s more straightforward and better takes into account how people self-identify.

  1. Consider a traditional lecture-based class with a fixed curriculum. That model is built around homogenity, not diversity: It’s much easier to operate if everyone involved has the same background, interests, learning-style, and rate of growth. Our model is the opposite: It’s helped not harmed by diversity.

  2. We haven’t previously tracked other demographic information in our admissions process, so we don’t have similar data for other groups, though we believe the same is true for them as well.

Nick bergson shilcock 150
Tweet

Paper of the Week: Reflections on Trusting Trust

This is part of our “Paper of the Week” series. For more info, check out our introductory blog post.

This week, we have two papers: a paper and a bonus meta-paper. We’ll start with the meta-paper.

How to Read a Paper

Hacker School alum Marcus Malka submitted How to Read a Paper by Srinivasan Keshav, a computer science professor at the University of Waterloo, and three of his students: Hossein Falaki, Earl Oliver, and Sumair Ur Rahman. It was originally published in the July 2007 issue of Computer Communication Review, but the version we’re linking to is a living document and was last updated in 2013.

How to Read a Paper has several useful tips and strategies for effectively reading academic papers. Some of it is geared towards researchers who are peer reviewing papers for their colleagues, but most of the tips are applicable for any reader.

Reflections on Trusting Trust

This week’s paper is Reflections on Trusting Trust by Ken Thompson. Thompson is the co-creator of the UNIX and Plan 9 operating systems as well as the Go programming language. The paper is adapted from a lecture he gave upon receiving a Turing Award for his hand in the creation of UNIX, and was published in the August 1984 issue of Communications of the ACM. It was submitted by Hacker Schooler Leah Hanson who shared the following:

A short piece by Ken Thompson illustrating that you have to trust your compiler/the people who wrote it: reading the source code [of the compiler, etc.] is not enough to be really sure that the compiler is doing what it should be doing. This had some of the mind-expanding feeling that recursion often gives people when they first grasp it; the malicious binary version of the compiler both affects the next version of the same compiler compiled with it (self-hosting compiler) and some target binary (the one that handles logging into Linux).

A recent related blog post, Countering “Trusting Trust” talks about a way to defend against this attack (given that not all of your compilers are infected by exactly the same attack). This method can have some false positives, in the form of compiler bugs (that are not malicious attacks).

Read Along

We want to let people participate in Paper of the Week, so we’re adding a section called Read Along. If you want to take part in Read Along, all you have to do is read the paper, create an original work in response, and email us a link of what you make by noon Eastern Time on the following Monday. Your original work might be a blog post explaining the paper or any insights it gave you, a program that implements some of the paper’s ideas, or something else entirely. It’s up to you. We’re not promising to publish everything, but our goal is to make Read Along something that’s easy to participate in, so don’t be shy about submitting, even if it’s something small.

This week, Hacker School facilitator Tom Ballinger decided to take on Guy Steele’s challenge to explain garbage collection using only words of one syllable as primitives. You can see the results on his blog.

Happy reading!

David albert 150
Tweet

Introducing “Paper of the Week”

Update 9/22/14: Hacker Schooler Marcus Malka suggested “How to Read a Paper” by Srinivasan Keshav as a companion to Paper of the Week. We’ve included it here for posterity. You can read more about it in PotW #2.

Today we’re introducing the first recurring series on the Hacker School blog. It’s called Paper of the Week. Every Monday we’ll link to a computer science paper that’s worth reading, submitted by a member of the Hacker School community.1

Our goal is to present academic computer science in a practical and accessible way. Reading the right papers can be enriching and enlightening, but there’s a lot of CS research happening and it’s hard to figure out what papers have useful insights into everyday programming. Lots of Hacker Schoolers read and talk about papers, and we thought it would be fun to share some of them with the rest of the world.

We’ll be featuring papers that are relevant to programmers of all levels. Some of them will be Ph.D dissertations, others will be short essays published in journals or just posted to the Web.

If you’re new to academic writing, the language can be dense and intimidating. This is true even for experienced programmers. Don’t despair! Sometimes you will need to read a sentence one word at a time, looking up the definition of each term you don’t understand. Often, it will take multiple reads to grok what the paper is about. Even if you don’t end up fully understanding a paper, the experience of puzzling through it is often worthwhile.

This week’s paper is Growing a Language by Guy Steele. It is a version of a talk he gave at OOPSLA ‘98. At the time, Steele was working at Sun Microsystems on the Java programming language. He was already well-known for his work developing Scheme with Gerald Sussman.

The paper was submitted by Hacker School resident Mel Chua who had this to say about it:

I ran into this during my first programming job as a fresh college grad. I didn’t have a formal CS education, so I didn’t know much about CS theory, and… to be entirely honest, at some level, I didn’t even want to look at it (because what if I looked and found it was Too Hard and I was Not Smart Enough To Be A Programmer?) So I irrationally avoided formal CS theory due to Impostor Syndrome.

However, I loved humor – which was how this paper snuck on my radar under the guise of “check out this great joke!” It wasn’t until partway through that I realized I was laughing at CS theory. Steele’s talk uses natural language to illustrate how programming languages are built up from primitives, but he does so in such a gentle, conversational way that it takes a little while to get the joke of what he’s trying to do. At first, it just sounds like he’s talking funny about topics that aren’t relevant to programming, but stay with it. It is absolutely worthwhile.

Happy reading!

  1. If this gets popular, we may also publish submissions from the public. No promises that we’ll ever do this, but if you feel the urge to send in your favorite paper, you can email potw@hackerschool.com.

David albert 150
Tweet

New batch names

Ever since we moved to overlapping batches, we’ve had a problem with our batch names. With eight batches a year, Summer, Fall, and Winter no longer make sense. For a while we’ve been avoiding the problem by embedding dates directly into the batch names, but that wasn’t an elegant solution.

Today we’re introducing new batch names. Batches will be named with a season, the number 1 or 2, and a year. For example, the batch starting on October 6th of this year is called Fall 2, 2014. Here is a full list of batch names for 2015:

  • Winter 2, 2015
  • Spring 1, 2015
  • Spring 2, 2015
  • Summer 1, 2015
  • Summer 2, 2015
  • Fall 1, 2015
  • Fall 2, 2015
  • Winter 1, 2015

It’s not a typo that Winter 2 comes before Winter 1. While it is confusing, we think the alternative isn’t much better: Winter 2, 2014 would be followed by Winter 1, 2015. We wish there was a perfect solution, but we feel a bit better knowing that naming is one of the two hard problems in Computer Science.1

Batch names can be abbreviated with the first letter of the season (e.g. F2'14). In the interest of disambiguation, the abbreviations for Spring and Summer use the first two letters of the season (Sp1'15 and Su2'16).

We’ve added the new batch names to our FAQ, apply page and the rest of our site. We’ve also retroactively applied the new names to all existing overlapping batches. If you find a place where we’re still using old batch names, please let us know.

  1. The others are cache invalidation and off-by-one errors.

David albert 150
Tweet

What people do at Hacker School

This post tries to answer a seemingly simple question: What do people work on at Hacker School? The question isn’t as simple as it first seems because Hacker School doesn’t have a curriculum and Hacker Schoolers are extremely diverse: They choose to do a huge range of things, from writing small games to contributing to large existing projects.

There are a few things that are true across the board. Everything written at Hacker School is open source, and everyone works on projects they choose for themselves based on their interests and what they want to learn. Most people focus on projects that will be hard but possible for them to do given their current programming abilities. Beyond that, it’s hard to make any general statements, and so the best way to get a sense for the types of things people do here is through examples.

So, without further ado, here are nine things people do at Hacker School:1

  • Explore a new programming language – with its creator. Daria has been learning Elixir, a functional programming language built on top of the Erlang virtual machine, with the help of its creator, José Valim, who’s currently in residence at Hacker School.

  • Implement the Fast Fourier Transform and build a custom compression scheme. Marisa and Nava have combined programming and linear algebra to do image processing and compression work.

  • Write a useful open source library, and become the top-trending project on GitHub. Sahat has been writing a token-based authentication module for AngularJS called Satellizer, and with nearly 2,800 stars it was recently the top-trending project on GitHub.

  • Contribute to Rust, and find bizarre bugs in its string search algorithm. Nick discovered that "bananas".contains("nana") returns false in Rust. He then tracked down the bug, submitted a patch, and wrote a blog post.

  • Do hard-core things with bytecode and add tail-call optimization to Python. Liuda wrote a Python decorator that can automatically make a function tail-recursive by manipulating its bytecode.

  • Build tools you want for yourself. Victor has been working on Ractor, a distributed actor system in F#.

  • Find serious security vulnerabilities, and make the web a little safer. Max discovered that Maven Central serves JAR files unencrypted by default, and wrote dilettante, a man-in-the-middle proxy that injects malicious code. His blog post received so much attention that the company behind Maven Central is reversing their policy and will start providing free HTTPS access.

  • Build a photobooth. Laura built everything for an ASCII photo booth, from the hardware to the software to process the images, and patched GPUImage’s Mosaic filter along the way. And a slew of Hacker Schoolers pitched in to make hardware hacks for our summer party.

  • Reflect and be introspective, especially about learning. In addition to working on projects, Hacker Schoolers reflect on how they learn. Dana wrote a great post about her experience at Hacker School.

If any of the above sounds exciting to you, you should apply to Hacker School. And please don’t let any of these projects intimidate you: Most people report that Hacker School enabled them to tackle things they previously thought they couldn’t.

  1. To be clear, this is just a small sampling of things Hacker Schoolers have done in the past few weeks, and isn’t in any way comprehensive.

Nick bergson shilcock 150
Tweet

Subtle -isms at Hacker School

We’ve enjoyed seeing our social rules spread into other parts of the tech community. Many people laugh in recognition when they hear about “No feigning surprise” or “No well-actuallys,” since they recognize that they’ve done these things and been annoyed when other people do them.

Our fourth rule, “No subtle -isms,” is less well-understood. Unlike the first three rules, it sounds vague and confusing, or maybe obvious – lots of organizations have a rule banning sexism. However, “no subtle -isms” doesn’t go without saying. Unlike many rules around sexism and racism, Hacker School’s social rule is intended to target subtle behaviors – things that nearly everyone does.

The social rules are lightweight

We often say that the Hacker School social rules are intended to be lightweight. Another way to put this is that we expect that Hacker Schoolers will occasionally break one of the social rules. When this happens, another Hacker Schooler will say, “Hey, that was a well-actually,” and the first Hacker Schooler will say “Oops, sorry!” That’s it – accidentally breaking social rules is common, expected, and readily forgiven. Even Hacker School founders sometimes slip up.

Hacker School social rules are much lighter than a code of conduct. Someone who violates a conference’s code of conduct could get written up, warned, or ejected from the conference. Violating a code of conduct is a big deal, and it usually isn’t hard to avoid doing so. By contrast, it’s much harder to avoid breaking the Hacker School social rules, and people often make mistakes.

Even the last rule is lightweight

Our last social rule, “No subtle -isms,” bans subtle racism, sexism, homophobia, transphobia, and other kinds of bias. Like the first three rules, it’s targeting subtle, accidental, mildly hurtful behavior. This rule isn’t targeting slurs, harassment, or threats. These kinds of severe violations would have consequences, up to and including expelling someone from Hacker School.

Breaking the fourth social rule, like breaking any other social rule, is an accident and a small thing. In theory, someone should be able to say “Hey, that was subtly sexist,” get the response “Oops, sorry!” and move on just as easily as if they’d well-actually'ed. In practice, people are less likely to point out when this rule is broken, and more likely to be defensive if they were the rule-breaker. We’d like to change this.

Following “No subtle -isms” looks different for different people. Someone who’s never thought much about bias in tech might decide to read Unlocking the Clubhouse or Stuck in the Shallow End. Someone else might check if they ask Hacker Schoolers of different races for help in proportion to the population at Hacker School. For faculty, since resident Lindsey Kuper mentioned it to us a year ago, we’ve been trying to stop using “you guys” to refer to mixed-gender groups. (As part of this effort, the very British facilitator Mary Rose Cook has started saying “y'all” with an English accent, which is a real treat for the rest of us.)

Like the other three rules, “No subtle -isms” is about recognizing the ways we’re unconsciously making our friends' lives a little worse. Like the other three rules, breaking it does not make you a bad person and is not a huge deal.

No debates, please

For the last year, the “No subtle -isms” rule has carried some implementation guidelines. One of these is asking people not to debate whether or not something is an -ism. From the manual:

If you see a subtle -ism at Hacker School, you can point it out to the relevant person, either publicly or privately, or you can ask one of the faculty to say something. After this, we ask that all further discussion move off of public channels. If you are a third party, and you don’t see what could be biased about the comment that was made, feel free to talk to faculty. Please don’t say, “Comment X wasn’t homophobic!” Similarly, please don’t pile on to someone who made a mistake. The “subtle” in “subtle -isms” means that it’s probably not obvious to everyone right away what was wrong with the comment.

When we introduced this policy, many women in our community responded positively. At the same time, some men described their process of discovering what their female friends were going through, and worried that other men would miss out on this opportunity to become allies. That’s a genuine cost of having this policy. Nevertheless, we believe that this cost is outweighed by two benefits. First, we want marginalized people to feel welcome, not like they have to defend their presence. Second, we don’t want marginalized people to have to spend time educating non-marginalized people who might be coming to these ideas for the first time. Faculty - not other Hacker Schoolers - are happy to help people discover resources for learning about anti-oppression.

To give everyone an equal chance to focus on programming, we ask that conversations about bias happen somewhere that’s opt-in (like a designated lunch discussion or a private email thread) and not somewhere that’s opt-out (like the mailing list, public chat, or within earshot of someone trying to program).

We’re far from perfect

We’re working to eliminate subtle -isms at Hacker School, but we still have a long way to go. For example, we’ve spent more time, energy, and money fighting sexism than fighting racism, homophobia, or transphobia, which is wrong of us. (Although our employees are diverse on some axes, we’re 90% white.) There are also class barriers to attending Hacker School - while Hacker School is free, living in New York for three months is not.

These are hard problems, but we’re committed to fixing them.

Join us

You’re welcome to use Hacker School social rules in your community. For an example of using the social rules in addition to a code of conduct, you can look at !!Con. If you use the rules, please credit and link to Hacker School.

Allison kaptur 150
Tweet

Welcome Rachel!

Rachel Vincent

Rachel

We’re excited to welcome Rachel Vincent to Hacker School!

Before joining us, Rachel worked in the fashion industry developing digital strategies, planning events, and maintaining e-commerce sites. At Hacker School, Rachel will be focused on our single biggest organizational weakness: operations. She’ll be responsible for keeping calendars and checklists, developing process out of chaos, and helping us automate as much of what makes Hacker School tick as possible.

Hiring Rachel is a milestone for us for a number of reasons. Not only is this the first non-programming position we’ve hired for, but Rachel is also the first person we’ve hired from outside the Hacker School community. Before Rachel, we had never written a job post, developed an interview process, or made a hiring decision without three months to get to know someone first.

We learned a lot during the process, which we plan on writing about in more detail soon. In the mean time, we want to thank everyone who applied and interviewed for the job. We are honored and humbled that so many highly qualified people wanted to work with us, and we’re sorry that we could only hire one person.

Please join us in welcoming Rachel to Hacker School! We’re thrilled to have her and we’re excited about all the ways that she’ll help make Hacker School better.

David albert 150
Tweet

Overlapping batches

Hacker Schoolers have helped to define a lot of what makes Hacker School great. This is true both in the obvious sense that Hacker School is fundamentally composed of the people who participate in it, but also in the deeper sense that they organize and create many of the things that make up the Hacker School experience.

When people come up with ways to improve Hacker School they don’t need anyone’s permission to see them through. If a Hacker Schooler wants to organize a seminar on Lua or get more regular code review, they can make it happen.

In past batches, people have started SICP and Nand2Tetris reading groups, set up the Hacker School Beverage Coalition, run Iron blogger challenges, and organized everything from poker nights to Code Review Thursdays.

But because we have month-long gaps between batches, we effectively throw out all of these additions to Hacker School at the end of each batch, and we never get to take advantage of the momentum we’ve built up.1 It’s also emotionally and psychologically taxing to always be starting fresh and to have to reestablish the social environment.

Beginning this summer, we’re going to try an experiment: We’re going to have overlapping batches, with a new batch starting every six weeks. This is different from what we’ve done for the past three years, which was to run three batches a year with one-month breaks in between. Batches will still be about three months long.

To start, each batch will have about 35 people in it. But since batches will overlap, each batch will get to spend about six weeks with the batch that started before it and about six weeks with the batch that comes after it.2 We’re calling each six-week period a rotation.

Overlapping batches

We think this will improve Hacker School in several ways.

First, more of the improvements people make to Hacker School will persist between batches. We think this will increase the rate at which Hacker School improves.

Second, Hacker Schoolers will still get a core group of people to bond tightly with — the 35 people in their batch — but they’ll also get to meet and work closely with more Hacker Schoolers than they otherwise would be able to.

Lastly, Hacker Schoolers starting their first rotation will benefit from being around the people who are doing their second rotation. The Hacker Schoolers who have already gotten the lay of the land will be able to help the newcomers more quickly acclimate to Hacker School’s environment.

  1. Starting fresh each batch does have at least one advantage: it lets us more easily recover from mistakes we’ve made.

  2. This won’t be true for our batch beginning on June 9th, since it will only overlap with the following batch. This means there will likely be fewer people at Hacker School during the first half of the batch compared to our past batch (~35 people instead of 55 people) and more people during the second half of the batch.

We’re hiring an operations person

Update 6/30/14: We’ve hired someone for this job. You can read more in our blog post “Welcome Rachel!

We want to hire an operations person. Up until now, everyone we’ve hired has been a programmer and a Hacker School alum. We’ve done many hundreds of programming interviews, but we haven’t interviewed for non-technical roles before, so this is uncharted territory for us. It’s also incredibly important: Whomever we hire for this role will be essential to our continued growth and success.

In much the same way that we think grades and exams are poor proxies for learning, we think many interviews are poor proxies for how people actually work and what companies are actually like. We want our interview process to give you a sense of what it’s like to work with us, and vice versa. Our goal is to have a process that accurately reflects our company.

We also want to clearly explain the expectations for this job and the details of our interview process ahead of time, so there aren’t surprises.

The job

We’re looking for someone to take charge of operations, accounting, and event coordination at Hacker School. This job will entail everything from managing our office space to handling inbound email to planning group alumni events to working with our lawyers and accountants. It will also entail working closely with the entire company (all eight of us!). Since we are a small team, everyone has significant responsibility and helps define and determine our future growth and success.

Who we’re looking for

We’re looking for someone who writes well, exercises good judgement, is extremely effective, and is energized by being around other people.

What we offer

  • Meaningful work, with a huge effect on people’s lives. To brag briefly: We’ve lost track of the number of alumni who have told us we changed their lives.
  • A friendly and intellectual atmosphere, and a tight-knit and supportive team of eight coworkers.
  • A warm and welcoming office in SoHo (Broadway and Grand — we’re near lots of good food and transit).
  • Potential for growth: If things go well, you will be COO.
  • A great health insurance plan. We have UnitedHealthCare Oxford Freedom and Hacker School pays 100% of the premium.
  • 15 days of vacation (we effectively have unlimited vacation, but we have a number to make sure people actually take it).
  • Complete organizational transparency: If we give you an offer, we will share all employee and founder salaries, how much cash we have, projected revenue, and the many risks we face. We will answer any questions you have about our company and prospects honestly and directly.
  • Speaking of transparency, we can pay between $50,000 and $60,000 a year, and we will give you stock options (~0.25%).

A few extra things to know

What to expect from our interview process

The first step is to email your resume or LinkedIn (just make sure it’s public) to ops@hackerschool.com, along with a paragraph or two about why you’re interested in the job. Please use thoughtful, conversational English.

Next, we’ll set up a phone call with one of the founders (Nick, Dave, or Sonali). This will be a short call to learn more about your background and interests, to share more about the job and Hacker School, and to answer any initial questions you have.

If that goes well, we’ll send you a short writing task, which will be representative of some of the work the job requires.

Lastly, we’ll schedule a half or full day of in-person interviews. This will be a series of interviews each with one or two Hacker School employees, as well as a lunch with several of us (we want you to get a chance to meet everyone in the company). We will do our best to make individual interviews representative of the actual work the job entails. For instance, we’ll tell you about a real challenge we’re facing, and you and the interviewer will work together to solve it. We’ll also give you lots of opportunities to ask us questions.

We think it’s rude when companies go silent and stop responding to people they’re interviewing. We promise not to do this, and will follow up with everyone who makes it to the phone round promptly with whether or not we’re interested in moving to the next round. Since we don’t know how much inbound interest we’ll get, we can’t promise to send personal responses to everyone who applies, but we will reply with a quick thank you and acknowledgement that we got your message.

We’ll do our best to give you an offer (or a no) within one week of your in-person interviews. We do not give exploding offers.

Mistakes we’ve made

We’ve made a lot of mistakes over the past three years of Hacker School. Sometimes our bad decisions are immediately obvious, but sometimes they’re more subtle and it’s only with time, feedback, and reflection that we realize we’ve screwed up. These mistakes are like poor architectural decisions that don’t manifest themselves until you’ve lived with them for a while.1

This is a post about mistakes we’ve made and tried to fix. In the interest of brevity, this post is greatly abridged, since the entire list would be much too long and embarrassing.

We no longer believe people must “love” programming to come to Hacker School. We learned that many of our alumni nearly didn’t apply because they worried they didn’t really “love” programming. Around the same time, Hacker School alumna Sunah Suh recommended the excellent book Unlocking the Clubhouse, which presents strong evidence that this language is gendered. We’ve since stopped saying that people must “love” programming (see my post about the word “hacker”).

We no longer tell people not to think about jobs until the end of the batch. The purpose of going to Hacker School is to be become a better programmer, and so we long advised people not to get distracted by jobs during the batch. But we found that this advice was hard for many people to follow, since most people quit their jobs for Hacker School, and it’s natural to think about what you’ll do afterwards. Given this, we now help people with jobs whenever they want and have optional interview prep sessions throughout the batch on Fridays. We’ve found that these changes have made it easier for people to focus on their education (and not worry about jobs) during the rest of their time at Hacker School.

We no longer have assigned facilitators or mandatory group dinners. These were two things we experimented with in the fall of 2012. At the time, they seemed like minor, positive changes, but we now see them as straying from our core principles. Dave wrote about this in detail in his post, Treating people like adults.

We don’t compare applicants to alumni who share superficial characteristics or demographics. This mistake is a bit different from the others because it’s not something we ever intentionally did. We realized about a year ago that we frequently found ourselves saying, “applicant X reminds me a lot of Y,” where X and Y more often than not shared the same race and gender, or other irrelevant demographics or traits. A similar voice or speech pattern can conjure up feelings, good or bad, about someone else and lead you to wrongly project other less superficial attributes onto that person. Given this, we now have a policy of not making comparisons betweens people who share superficial characteristics and call each other out any time we do.2

Our interview process now includes a pairing session. For our first five batches, our interview process didn’t include any programming or time working with applicants. In retrospect, this seems obviously wrong: One of the biggest questions we try to answer in our admissions process is what it’s like to work with someone, and yet we were never working with them at any point in the process. We’ve since tried to fix this by turning our second round interview into a pairing session, where the goal is to treat applicants as if they were already working with us at Hacker School. They get to choose the language to use and the project to work on, as well as what specifically they want to do (e.g., to add a small new feature, refactor a chunk of their code, etc.). Many applicants have told us that they really enjoyed the pairing sessions, and benefited from it regardless of whether they got in.

We shouldn’t have named our company “Hacker School.” Both parts of our name have caused us trouble: Hacker because so many people take it to mean a person who breaks into computers rather than a clever programmer.3 School because it implies a rigid and traditional approach to education that we emphatically reject. This mistake is different from the others on this list because we haven’t yet corrected it, and probably never will, given how time-consuming and costly it would be.

  1. If only we could wave a wand and discover the things we do now that we’ll someday consider mistakes!

  2. We’re also updating our internal review system to obfuscate applicants' names to avoid any subconscious bias during our application review process. We’ll soon be considering applicants named “Blue Dart” and “Purple Rover” rather than “Jane Doe” and “José Smith”.

  3. We’ve also struggled to a lesser extent with the stereotype of “nerdy white guys” that many people associate with the word “hacker.” See our post, What we mean when we say “hacker”.

Summer and fall 2014 applications are open

Applications for both the summer 2014 and fall 2014 batches of Hacker School are now open. Batches are filled on a rolling basis and applications close either two weeks before the batch begins or when the batch fills up,1 whichever happens first, so the earlier you apply, the better. If we admit you to Hacker School but the batch you apply to is already full, you’ll have the option of deferring to a later batch. Before applying, we strongly encourage you to read our about page, FAQ and User’s Manual.

This is the first time we’ve opened applications for multiple batches at once. Traditionally, we’ve opened applications two to three months before each batch starts. Opening applications for the fall 2014 batch (and every subsequent batch) at least six months ahead of time and doing interviews on a rolling basis will give people who need more than a few months of heads-up the chance to do Hacker School. Knowing the size of future batches further in advance gives us more certainty and allows us to plan for upcoming batches more effectively.

Because we’ve changed to rolling admissions, we no longer have fixed application deadlines. While it’s better to apply earlier, don’t stress too much about delaying for a few days, especially if we’ve recently opened applications for the batch you’re applying to. We don’t believe the application should take long to finish, but it’s better to submit a good quality application than a rushed one.

Good Hacker School applications are well written. Being able to write well isn’t an official qualification for Hacker School, but reality has made it a de facto one. As we get more applications, we have less time to read each one. This means that if you’d make an awesome Hacker Schooler but it doesn’t come across in your application, there’s a good chance that we’ll miss you. Writing clearly will help us see that you’re a friendly, self-directed, intellectually curious and rigorous person. You can find out more about what we look for in Hacker Schoolers in an older blog post.

Hacker School is a place to become a dramatically better programmer, whether you’ve been programming for 20 years or just for a few months. If the idea of spending three months with like minded people building an OS kernel, a pattern matching library, a BitTorrent client, a game engine, or anything else that strikes your fancy sounds exciting, Hacker School is for you. If you don’t know what any of those things are, don’t worry. Hacker School is for you too.

At Hacker School, we value intrinsic motivation and self-direction. Hacker School is free for everyone. The environment is supportive, gender-balanced, and intellectual. If hearing this gets you excited, we’d love to have you apply.

  1. Our main constraint is physical space. If we accept enough people into a future batch, we will move to a larger location.

David albert 150
Tweet

Acceptance rates

In August of 2012, we made the mistake of sharing how many people had applied to our summer batch, and with that we implicitly shared Hacker School’s acceptance rate. We now believe this was a bad decision and we no longer publish our admissions statistics.

That’s because we think about admissions differently than much of the world. While most colleges and universities want to make their acceptance rates as low as possible, we’d be thrilled if we could increase ours. We think there are several reasons why low acceptance rates are bad, and few reasons why they’re good for us.1

Our acceptance rate is the complement of our rejection rate, and our rejection rate is a direct measure of how much time we spend reviewing and interviewing applicants who we conclude aren’t good fits for Hacker School. Why would we try to spend more of our and others' time doing that? We currently spend a few hundred hours each batch on admissions, and our goal is to reduce this number, not increase it.

There are two ways to make your acceptance rate go down: You can either get more unqualified applicants to apply (and then reject them) or reject qualified applicants who you would have otherwise accepted. Neither of those options make any sense to us. Our goal is to get more qualified people to apply and then accept them.

In our experience, bragging about how few people you accept has the negative effect of encouraging people to apply who are just looking for something impressive to put on their resumes, and who don’t actually have a good sense of what Hacker School is and why they want to do it. Worse, it can discourage qualified people from applying because they’re convinced they’ll never get in. This scares us, since the latter group is thoughtful enough to pause and reflect on whether they and Hacker School are a good fit. And those are much more likely to be the people who would do well at Hacker School.

  1. There is one case when it’s clearly been helpful for us to have a low acceptance rate: When talking with companies about recruiting Hacker School alumni. Companies are always happy to hear we have a low acceptance rate, presumably because they think it’s a good proxy for the quality of our alumni. We’d happily do away with this advantage in exchange for a higher acceptance rate and more qualified people at Hacker School.

    Traditional colleges and universities have other reasons for wanting low acceptance rates, for example, because it boosts their rankings on lists of the “best” colleges. We don’t – and, thankfully, don’t need to – care about such things.

Winter 2014 residents

Ever since we ran our first experimental residents program in the fall 2012 batch, residents have become an institution at Hacker School. We’re delighted to welcome another exciting crop for the winter 2014 batch.

Five of our former residents are joining us again: Mel Chua, the rare engineering education expert who’s also an engineer herself; Stefan Karpinski, a co-creator of Julia who’s gotten Hacker Schoolers hooked on LLVM, dynamic dispatch, and the intricacies of floating-point numbers; Lindsey Kuper, a researcher on the cutting edge of deterministic parallel computing; Kevin Lynagh, a prolific ClojureScript and JavaScript hacker with one of the coolest tattoos we’ve ever seen; and Jessica McKellar, a Twisted maintainer, Python core contributor, and talented nerd-sniper.

Three new residents are joining us as well. Evan Czaplicki is the creator of Elm, a purely functional language for web programming. Greg Price is a kernel hacker, programming language enthusiast, and systems programmer who’s worked at Ksplice and Quora. Robert Lefkowitz, aka r0ml, gave a talk about APL during the last batch of Hacker School that had more Q&A and more laughter than any other talk we’ve had.

The Maintainers Program didn’t work out as well as we hoped last batch, and we’re discontinuing it (in its current form, anyway). We found that for Stefan, being a maintainer was just like being a resident—he was already helping Hacker Schoolers contribute to Julia. For Glyph and David, Hacker Schoolers weren’t very interested in contributing to their projects. We’re still interested in running experiments that help Hacker Schoolers contribute to free and open source software, and we’ll have more experiments in the future.

We want to create an environment where all Hacker Schoolers can become much better programmers over the course of the batch. Residents help make sure that even the most advanced Hacker Schoolers—many of whom have more expertise than facilitators—get exposed to new ideas, have their thinking challenged, and work with specialists in industry and academia. We can’t wait to see what the winter batch will do with this round of residents.

Allison kaptur 150
Tweet

Words are hard

We don’t like to use the term “student” to refer to people who come to Hacker School, but we’ve had trouble finding another word to replace it.

Words are hard to choose because they come with context. In some ways, “student” is a good word for us to use. Almost everyone who comes to Hacker School can identify as having been a student at some point in their lives. This makes “student” friendly and familiar. Calling yourself a student can also signal your commitment to learning.

Unfortunately, “student” brings some non-Hacker Schooly context with it. “Student” implies a traditional student-teacher relationship that isn’t present at Hacker School. It implies that there is someone else responsible for your education, when there is not.

We’ve tried using other words, but none of them are perfect. A safe one is “Hacker Schooler.” All it implies is that you’ve decided to come to Hacker School. Unfortunately “Hacker Schooler” can be a bit of a mouthful and cumbersome to use in some situations.

Another term we’ve considered using is “hacker.” The word “hacker” describes a very particular identity and subculture and there are plenty of people who make great Hacker Schoolers but don’t identify as hackers.

The word “developer” has similar problems, but for different reasons. It has a professional connotation similar to “software engineer,” but being a professional programmer, or wanting to be one, is not a requirement for coming to Hacker School either.

“Programmer” is actually not bad. All that it implies is that you write computer programs. Everyone at Hacker School does that. Still, even “programmer” has its problems. Not everyone who applies to Hacker School identifies as a programmer when they apply.

“Student” isn’t the only word that we struggle with. “School” has given us even more trouble and it has many of the same problems. A school is a place of learning, which is a good start, but it also has teachers, grades, homework, and curriculum, none of which are part of Hacker School. Explanations of Hacker School often start by explaining that it’s probably not like any school you’re familiar with.

Calling ourselves a school causes all sorts of confusion. Experienced programmers think they have too much experience for Hacker School even though Hacker School is designed for people with a wide range of experience levels,1 our alumni have trouble describing what Hacker School is like to other programmers, and companies have a hard time understanding how they should compare us to other places they hire from.

This seems to be a problem for anyone trying to explain the unfamiliar. If you invent new words, few people will know what you’re talking about, but if you make analogies using existing words, you bring along all their context, whether you want to or not.

  1. There are also new programmers who think they don’t have enough experience for Hacker School, though this probably doesn’t have to do with the word “school.”

David albert 150
Tweet

Google sponsors over $150k of grants for female programmers

We’re excited to announce that Google is sponsoring over $150,000 of need-based grants for female programmers to come to Hacker School. These grants will allow us to provide financial assistance for living expenses to women for our upcoming batch and hopefully beyond. (Tuition, of course, remains free for everyone; the grants are only for living expenses for those in financial need.1)

This is the largest sponsorship of grants we’ve ever received, and we’re incredibly thankful for Google’s support. For nearly two years, we’ve worked to secure grants for female programmers who cannot afford to live without an income during Hacker School. Getting enough grants has been (and remains) a challenging and time-consuming process. In addition to helping make Hacker School bigger and better, Google’s generous support will give us a bit of breathing room over the next months, and allow us to spend even more time on improving other parts of Hacker School.

Thank you, Google!

Google
  1. You can read more about our thinking behind these grants and how they work in some of our previous blog posts.

Treating people like adults

Treating people like adults is a core tenet of Hacker School. That’s because trusting people to make good decisions for themselves is a huge part of self-directed education. In the fall 2012 batch, we made two small, seemingly innocuous changes to the structure of Hacker School that had the unintended effect of not treating people like adults. The changes had a surprisingly large, negative impact on Hacker School. Here’s what happened and what we do differently now.

The first change we made was giving each facilitator a group of Hacker Schoolers to get to know over the course of the batch. We did this because we wanted to develop a good mental model of each programmer at Hacker School, and make sure no one slipped through the cracks. The outwardly visible changes were minimal. Everyone got assigned to a morning checkin group with their facilitator, and each facilitator met personally with each Hacker Schooler in their checkin group around once a week.

In the previous batch, each facilitator had been assigned to one of our three rooms, and Hacker Schoolers tended to go to the same room every day out of habit. Checkin groups were large—nine or ten people to a group, two groups per room, one facilitator per group—so this new arrangement felt pretty similar.

The second change we made was suggested by Hacker Schoolers from the previous batch. Since the beginning of Hacker School, we’d had a mandatory, large group dinner every week. Twice in the summer 2012 batch, we experimented with splitting up into smaller groups and going to restaurants for dinner as a change of pace. Everyone seemed to like the smaller dinners because they were more intimate. At the end of the batch, a number of people told us that we should make all dinners small instead of large, so we decided to give it a shot.

Both of these changes felt minor, but their side effects were largely negative and they didn’t do a good job at accomplishing their goals.

Knowing that everyone had a facilitator assigned to them introduced a different power dynamic and encouraged people to treat us like teachers and to not take responsibility for their own learning. Checkins were delivered to facilitators and felt a lot like reports on homework assignments (which we don’t have). A lot of people didn’t believe checkins were valuable, so we had trouble convincing everyone to show up to Hacker School on time. Our mental model of everyone did get better, but the negative effects of the change far outweighed the positive ones.

Assigned small group dinners felt like class trips and we felt like chaperones. We would say things like “Is everyone here? OK, let’s go!” The class trip dynamic extended to the dinners themselves: they didn’t feel like relaxed meals between peers and weren’t the bonding experiences that we hoped they would be.

Since last fall, we’ve made changes to checkins and dinners to make sure that we’re treating everyone at Hacker School like an adult.

Facilitators no longer have assigned Hacker Schoolers. Checkin groups are now half the size and often don’t contain any facilitators. Hacker Schoolers are responsible for structuring their checkins however they see fit. This means we don’t have as good a mental model of every Hacker Schooler, but it turns out that in an environment where everyone is responsible for their own education, having a good, centralized mental model of each person is much less important than we originally thought.

Dinners are now in one big group, not at restaurants, optional, and always accompanied by a talk. Having large non-restaurant dinners eliminates the chaperoning, making them optional trusts each person to decide whether or not they should come, and having talks makes them worth coming to.

There’s also been a more fundamental change. Since last summer, it feels like we’ve also become a happier organization. It’s impossible to prove the connection, but I feel like it has a lot to do with running Hacker School in a way that’s more in line with our values.

David albert 150
Tweet

Fundamental qualities of good programmers

At Hacker School, we spend a lot of time thinking about what it means to be a good programmer. My goal with this post is to share some of our thinking on this topic and to hopefully help people better understand Hacker School in the process.

First, a quick note: This is just one way to think about becoming a better programmer. It’s almost certainly not the only way to think about it and you may find some other way more useful to you.

This is a list of what we believe are fundamental qualities that all good programmers share. It is based on our observations from the more than two years that we’ve been running Hacker School. It’s not perfect—there are qualities that we’re missing and there’s some overlap between the ones that we have listed—but it’s been very useful for us so far. We expect that it will grow and change as Hacker School does.

Knowing one programming language really well

Programming languages are just tools for telling computers what to do. It is better to have a strong command of one than a weak command of a bunch. Good programmers have at least one language that they know inside and out and can reach for to easily solve whatever problem is at hand.1

Being a systematic debugger

Being a systematic debugger means that you have a good mental model of your code and that when you run into a bug—that is, when your program doesn’t work as expected—you generate hypotheses about what’s wrong, instead of blindly changing things until your program works.

Having a good mental model of your programming environment

Most programs interact with the outside world, so having an understanding of your environment is important. This includes the I/O and concurrency primitives that your language provides, the way your language finds, loads, compiles, and runs code, the way that your program gets info from the outside world (e.g., environmental variables and command line arguments), and the way your OS handles file access, device access, search paths, etc.

Having a good mental model of the hardware you use

Knowing how your hardware works makes it easier to write efficient programs. Even if you are writing programs in a higher level language, understanding things like the call stack, the MMU, the cost of context switching, the memory hierarchy, and the characteristics of the network you are connected to will inform your programming decisions.

Being comfortable with algorithmic thinking

A lot of people confuse algorithmic thinking with knowing a bunch of particular algorithms like quicksort or binary search.

An algorithm is a set of steps describing a calculation. Much of programming is simply describing algorithms to a computer. Being comfortable with algorithmic thinking means having good intuition for how to store and manipulate your data, being able to think both iteratively and recursively, and being able to reason about the performance characteristics of the code you write.

Being comfortable with mathematical thinking

Programming is not as math-heavy as many non-programmers think, but as you start tackling tougher problems, you’ll find that being comfortable with math comes in handy. Many interesting areas of programming like computer graphics, signal processing, and cryptography require a deep understanding of math. Even just measuring the performance of your code can require some statistical thinking.

Being able to write a program from scratch

It is hard to imagine a good programmer who cannot write a program from scratch.

Being able to work on a small piece of a large program

Most of the time you’re not writing a program from scratch. Instead, you’re working with other programmers on a large project.

Knowing how to structure your code

Well-structured code allows you to easily navigate up and down through layers of abstraction. In a well-structured program, you can ignore implementation details when they don’t matter and modify the implementation without having to make changes in all the places where the code is used.

Having a large code radius

Code radius is a term that Alan came up with. Your code radius is the size of the largest program you’re comfortable writing from scratch. Increasing your code radius takes practice. The structural tools that work well for a 500 line program are not necessarily the same as the ones that work for a 5,000 line program or a 50,000 line program.

Being able to code quickly

Often times it’s faster to try out a few different solutions and decide which is the most elegant than to figure out the correct one just by thinking about it. If you find “quickly” to be too subjective, consider the inverse: It’s hard to imagine a good programmer who is slow.

Being productive with your tools

Good programmers use their tools effectively. This doesn’t mean you have to use all the tools available to you in order to be a good programmer—not everyone likes IDEs and plenty of good programmers prefer printf to a debugger—just that good programmers are productive with the tools they choose to use.

  1. This is not to say that all programming languages are equivalent, that they are equally good at all tasks, or that it isn’t worth learning more than one programming language. Just that all good programmers have a deep understanding of at least one language.

David albert 150
Tweet

Dropbox, Etsy, Jane Street, Tapad, and Tumblr support female programmers at Hacker School

We’re happy to announce that Dropbox, Etsy, Jane Street, Tapad, and Tumblr have all sponsored grants for female programmers for our current batch of Hacker School. Each grant is for $7,000, and makes it possible for a woman to attend who would otherwise not be able to afford to (Hacker School remains free for everyone, so the grants are used for food, housing, and other living expenses during the batch).

This support has helped make our current batch 42% women. These grants make Hacker School bigger, more diverse, and a better experience for everyone involved.

These grants are our hack for how to have a gender-balanced environment without negatively impacting men or lowering the bar for women. The grants work because they increase the pool of qualified women who are able to do Hacker School.

We also believe the grants help signal to women and men around the world that Hacker School is institutionally and structurally committed to having a gender-balanced environment. This helps attract phenomenal people who don’t need financial assistance, in addition to the people who do.

One of the many benefits of having a gender-balanced environment is that, at least within the confines of Hacker School, the pressure to represent or focus on “women in programming” largely fades away, and people are free to focus on programming rather than rehashing tired arguments.

In fact, one of our social rules is that Hacker School is not a place to discuss gender or sexism in tech. This is because the purpose of having a gender-balanced environment is to have a place where women can simply be programmers, rather than forever being seen as “female programmers” or experiencing stereotype threat. There are many places in the world to discuss and debate these issues, but there are precious few where people can avoid them. We want Hacker School to be one of those places (read more in our User’s Manual). We’ve heard from many women that this was one of the most refreshing aspects of Hacker School.

On behalf of all of Hacker School, we’d like to thank Karen Sperling at Dropbox, Collette Ricard and Kellan Elliott-McCrea at Etsy, David Powers and Caitlin Maynard at Jane Street, Michael Moss at Tapad, and Sean McDermott at Tumblr, who have all been essential to making this possible. Thank you!

Who comes to Hacker School?

People frequently ask us who comes to Hacker School. They usually have some preconceived notions about what the “typical” Hacker Schooler looks like, and they’re often surprised when they learn who our alumni actually are.

Much of this confusion comes from the fact that Hacker School is incredibly diverse. There are now about 300 Hacker Schoolers from our first two and a half years, and they come from a staggering array of backgrounds. For most demographic questions, the answer to “Have you had any Hacker Schoolers who __?” is probably “Yes.”1

The goal of this post is to dispel some common myths about who comes to Hacker School. The following data comes from our current batch (fall 2013), which is our eighth batch and has 59 people in it. Fifty seven people (97% of the batch) responded to our survey.2

Myth #1: Hacker School is only for new programmers

False. While we’re very much open to relatively new programmers, nearly two-thirds (62%) of our current batch had been programming for at least two years prior to coming to Hacker School, and 41% of the batch had been programming for more than three years.

Additionally, nearly half of the current batch (45%) worked professionally as programmers prior to Hacker School, and 42% of those (19% of the entire batch) had at least three years of professional programming experience. Just under a quarter (22%) have undergraduate computer science degrees, and half of those (11% of the entire batch) have graduate degrees in computer science.

Myth #2: Hacker School is only for experienced programmers

False. More than a third (37%) had been programming for less than two years; 8% had been programming for less than six months.

Just over half (55%) have never worked professionally as programmers, and more than three quarters (78%) don’t have computer science degrees.

Myth #3: Hacker School is only for people in New York

False. Most people in the current batch (72%) moved to New York for Hacker School.

And people aren’t just coming from neighboring states. The current batch hails from California, Colorado, Georgia, Kentucky, Massachusetts, Michigan, New Jersey, Oregon, Pennsylvania, Texas, Virginia, Washington, and Wisconsin. There are even a few from New York.

More strikingly, 29% of the current batch came from outside the US. This batch includes Hacker Schoolers from more than a dozen countries, including Belgium, Canada, France, Hong Kong, Italy, Japan, Norway, Russia, Sweden, Switzerland, UK, and Ukraine.

Myth #4: Hacker School is only for young people and recent college grads

False. The average age in the current batch is just over 28, with ages ranging from 19 to 58. There are more people in the current batch over 30 than under 25.

  1. There are of course characteristics that Hacker Schoolers have in common. This is unsurprising, since we look for smart, friendly, self-directed, intellectually curious people who enjoy programming and want to get dramatically better.

  2. We didn’t have this data until very recently. Since we don’t make admissions decisions based on where people live, how old they are, or what degrees they have, we don’t ask any of these questions on our application. We had to survey our batch to write this post.

Fall Residents at Hacker School

This summer’s residencies have been our most successful so far. Residents and Hacker Schoolers worked together to write interpreters and compilers, learn about their learning styles, and compete in the ICFP contest, along with dozens of other projects and collaborations.

We’ve written previously about the minimum amount of experience you need to come to Hacker School. Sometimes people ask the opposite: can you have too much experience for Hacker School to be a good fit? We believe there isn’t an upper bound. Even experienced programmers can improve with three months of focused, deliberate practice in a supportive community. Residents help us build that community, so that even the most experienced Hacker Schoolers have a chance to work with someone who will blow their minds. This summer’s ten smart, knowledgeable, and kind residents were a tremendous asset to Hacker School and set a high bar for future residents.

With this backdrop, we’re delighted to welcome eight residents this fall. Mel Chua and Lindsey Kuper are coming back for their second residencies, Jessica McKellar is coming back for her fourth, and Will Byrd, Philip Guo, Jacqui Maher, Yaron Minsky, and Alex Rudnick will be joining us for the first time.

We’re equally excited to have three great previous residents – Stefan Karpinski, Glyph, and David Nolen – kicking off our Maintainers Program.

You can read more about our residents, and there’s still (barely) time to apply for the fall 2013 batch.

Allison kaptur 150
Tweet

Announcing the Hacker School Maintainers Program

Today we’re announcing the Hacker School Maintainers Program. The Maintainers Program will bring maintainers of free and open source software (FOSS) into Hacker School, either physically or virtually, to help Hacker Schoolers contribute to their projects. We’re extremely excited to have Stefan Karpinski, David Nolen, and Glyph as our first maintainers.

We’re running this experiment for three reasons. First, we think it will be a useful piece of optional structure for some Hacker Schoolers. Second, we would like to give back to FOSS communities, many of which we owe a great deal to. Finally, we would like to experiment with running parts of Hacker School asynchronously over the Internet.

Optional structure

We often get asked “How do you learn at Hacker School without any structure?” This is a confusing question for us because we don’t believe structure is inherently bad. We believe people learn best when they are driving their own education and studying things that interest them. That doesn’t mean that we don’t think structure plays a part in the learning process, just that it must be self-imposed.

Few people can learn without structure, but the pieces of structure that work for you may be different than the ones that work for someone else. Furthermore, coming up with structure on your own is hard. This is why we spend so much time thinking about how we can provide optional structure at Hacker School. We like optional structure because it allows you to pick the pieces that work for you and ignore everything that doesn’t.

We’ve been bitten by having too much required structure before. One year ago, in our summer 2012 batch, we held an Open Source Week, where for one week everyone stopped what they were working on and contributed to FOSS projects. Some Hacker Schoolers loved it and got a lot out of it. Others would have rather stayed home.

Contributing to FOSS isn’t right for everyone. Some people find it uninteresting. For others, it simply isn’t the best way for them to grow as a programmer. But for those for whom the time is right, contributing to FOSS projects can be a very important step in their evolution as programmers and we’re excited to make it a larger part of Hacker School.

Giving back

Hacker School wouldn’t be possible without free software. We would like to find ways to give back to the communities and projects that have made our lives better. Many of us still remember what it was like to have to pay hundreds of dollars for a C compiler. Now it’s hard to imagine a world where we don’t have the freedom to read and modify the source of the tools we use every day. We’re not sure what sort of impact the Maintainers Program will have, but we won’t consider it a success unless the projects and maintainers we work with do, too.

Asynchronous Hacker School

Hacker School would be far worse if it didn’t take place in person. Friendship, comfort and trust, all of which are necessary for effective learning, are more easily built in person than online. This means that many of the benefits afforded to internet businesses are not available to us. We’d like to find ways to use the Internet to our advantage without sacrificing the things that make Hacker School great.

As an experiment, we’ve designed the Maintainers Program to work even if a maintainer can’t physically be at Hacker School. We’re going to use all the software we have at our disposal—video chat, ssh, screen, tmux, mailing lists, and more—to make the Maintainers Program work as seamlessly as possible whether or not the maintainer is in the same room or even available at the same time.

How does it work?

The Maintainers Program is simple in practice: contributing to FOSS projects can be hard, especially if you’ve never done it before. Maintainers will help Hacker Schoolers navigate their project, understand what needs to be done, and give feedback and code review. There are no guarantees that your code makes it into a project, but we hope that having maintainers available for guidance will make it more likely.

We want our maintainers to be genuinely nice people. This is why all of our initial maintainers are past residents. Stefan has been a resident every batch since we started the Residency program. He’s a great evangelist for the Julia programming language and because Julia is written almost entirely in Julia, the path from user to contributor is a smooth one. David was a resident in our fall 2012 batch and he’s been a constant presence around Hacker School ever since. His enthusiasm for ClojureScript and core.async is infectious and he’s been instrumental in helping Hacker Schoolers contribute to open source projects in the past. Glyph is the founder of the Twisted project and an incredibly warm person. He was a resident in our winter 2013 batch and we’re thrilled that we could find a way to get him involved with Hacker School again.

We’re also looking for a few more maintainers. We’re going to be asking people we know for recommendations, but we’d also love to have your input. We’re looking for maintainers who are warm and empathetic and who have projects with existing communities of contributors. If you think of someone (or want to nominate yourself), you can use this Google form.

As much as we’d like to respond to everyone who nominates someone, we simply don’t have the time, so don’t expect a response unless we have questions about the person you’re nominating.

We’re really excited to be welcoming Stefan, David, and Glyph back for another batch of Hacker School. If contributing to Julia, ClojureScript, core.async or Twisted is something that you’d enjoy doing, there’s still time to apply for our fall 2013 batch.

David albert 150
Tweet

Partnering with Juniper

I’m excited to announce that we’ve partnered with Juniper Networks.

Juniper has been a wonderful supporter of Hacker School: They’ve both sponsored our summer 2013 batch and provided 20 $5,000 grants for female programmers. Although Hacker School is free for all students, living in New York for three months is most certainly not, and these grants made a huge difference for a number of women who would otherwise have been unable to attend Hacker School.

Throughout our work together, Juniper has proven itself to care deeply about supporting programmers, and increasing the number of female developers in particular. This support helped make our summer 2013 batch our largest yet with 70 students.

This batch has also been one of our most diverse: we welcomed robotics and statistics Ph.D’s, a former professional ballet dancer, a 20-year veteran of Wall Street, our first high school student, and bright and experienced engineers from some of the world’s top startups and tech companies. Seventy-five percent of our students came from outside of New York, 15% came from outside of the country, and 30% were female.

Hacker School works best with such a diverse student body. Our model is based in large part on people coming together from a range of backgrounds and experience levels, and helping each grow as programmers. Juniper’s support this summer has helped make this possible.

Thank you, Juniper!

Juniper logo

Little Lisp interpreter

Little Lisp is an interpreter that supports function invocation, lambdas, lets, ifs, numbers, strings, a few library functions, and lists. I wrote it for a lightning talk at Hacker School to show how easy it is to write an interpreter. The code is 116 lines of JavaScript. I will explain how it works.

First, let’s learn some Lisp.

Basic Lisp

This is an atom, the simplest Lisp form:

1

This is another atom, a string:

"a"

This is an empty list:

()

This is a list containing an atom:

(1)

This is a list containing two atoms:

(1 2)

This is a list containing an atom and another list:

(1 (2))

This is a function invocation. A function invocation comprises a list where the first element is the function and the rest of the elements are the arguments. first takes one argument, (1 2), and returns 1.

(first (1 2))

 => 1

This is a lambda, which is a function definition. The function takes a parameter, x, and just returns it.

(lambda (x)
 x)

This is a lambda invocation. A lambda invocation comprises a list where the first element is a lambda and the rest of the elements are the arguments. The lambda takes one argument, "Lisp", and returns it.

((lambda (x)
  x)
 "Lisp")

 => "Lisp"

How Little Lisp works

Writing a Lisp interpreter is really easy.

The code for Little Lisp has two parts: the parser and the interpreter.

The parser

Parsing has two phases: tokenizing and parenthesizing.

tokenize() takes a string of Lisp code, puts spaces around every parenthesis and splits on whitespace. For example, it takes something like ((lambda (x) x) "Lisp"), transforms it into ( ( lambda ( x ) x ) "Lisp" ) and transforms that into ['(', '(', 'lambda', '(', 'x', ')', 'x', ')', '"Lisp"', ')'].

1   var tokenize = function(input) {
2     return replace(/\(/g, ' ( ')
3           .replace(/\)/g, ' ) ')
4           .trim()
5           .split(/\s+/);
6   };

parenthesize() takes the tokens produced by tokenize() and produces a nested array that mimics the structure of the Lisp code. Each atom in the nested array is labelled as an identifier or a literal. For example, ['(', '(', 'lambda', '(', 'x', ')', 'x', ')', '"Lisp"', ')'] is transformed into:

[[{ type: 'identifier', value: 'lambda' }, [{ type: 'identifier', value: 'x' }],
  { type: 'identifier', value: 'x' }],
 { type: 'literal', value: 'Lisp' }]

parenthesize() goes through the tokens, one by one. If the current token is an opening parenthesis, it starts building a new array. If the current token is an atom, it labels it with its type and appends it to the current array. If the current token is a closing parenthesis, it stops building the current array and continues building the enclosing array.

1   var parenthesize = function(input, list) {
2     if (list === undefined) {
3       return parenthesize(input, []);
4     } else {
5       var token = input.shift();
6       if (token === undefined) {
7         return list.pop();
8       } else if (token === "(") {
9         list.push(parenthesize(input, []));
10        return parenthesize(input, list);
11      } else if (token === ")") {
12        return list;
13      } else {
14        return parenthesize(input, list.concat(categorize(token)));
15      }
16    }
17  };

When parenthesize() is first called, the input parameter contains the array of tokens returned by tokenize(). For example:

['(', '(', 'lambda', '(', 'x', ')', 'x', ')', '"Lisp"', ')']

When parenthesize() is first called, the list parameter is undefined. Lines 2-3 run and parenthesize() recurses with list set to an empty array.

In the recursion, line 5 runs and removes the first opening parenthesis from input. Line 9 starts a new, empty list by recursing with a new, empty array.

In the recursion, line 5 runs and removes another opening parenthesis from input. Line 9 starts another new, empty list by recursing with another new, empty array.

In the recursion, input is ['lambda', '(', 'x', ')', 'x', ')', '"Lisp"', ')']. Line 14 runs with token set to lambda. It calls categorize() and passes lambda as the input argument. Line 7 of categorize() runs and returns an object with type set to identifier and value set to lambda.

1   var categorize = function(input) {
2     if (!isNaN(parseFloat(input))) {
3       return { type:'literal', value: parseFloat(input) };
4     } else if (input[0] === '"' && input.slice(-1) === '"') {
5       return { type:'literal', value: input.slice(1, -1) };
6     } else {
7       return { type:'identifier', value: input };
8     }
9   };

Line 14 of parenthesize() appends to list the object returned by categorize() and recurses with the rest of the input and list.

1   var parenthesize = function(input, list) {
2     if (list === undefined) {
3       return parenthesize(input, []);
4     } else {
5       var token = input.shift();
6       if (token === undefined) {
7         return list.pop();
8       } else if (token === "(") {
9         list.push(parenthesize(input, []));
10        return parenthesize(input, list);
11      } else if (token === ")") {
12        return list;
13      } else {
14        return parenthesize(input, list.concat(categorize(token)));
15      }
16    }
17  };

In the recursion, the next token is a parenthesis. Line 9 of parenthesize() starts a new, empty list by recursing with an empty array. In the recursion, input is ['x', ')', 'x', ')', '"Lisp"', ')']. Line 14 runs with token set to x. It makes a new object with a value of x and a type of identifier. It appends this object to list and recurses.

In the recursion, the next token is a closing parenthesis. Line 12 runs and returns the completed list: [{ type: 'identifier', value: 'x' }].

parenthesize() continues recursing until it has processed all of the input tokens. It returns the nested array of typed atoms.

parse() is the successive application of tokenize() and parenthesize():

1   var parse = function(input) {
2     return parenthesize(tokenize(input));
3   };

Given a starting input of ((lambda (x) x) "Lisp"), the final output of the parser is:

[[{ type: 'identifier', value: 'lambda' }, [{ type: 'identifier', value: 'x' }],
  { type: 'identifier', value: 'x' }],
 { type: 'literal', value: 'Lisp' }]

The interpreter

After parsing is complete, interpreting begins.

interpret() receives the output of parse() and executes it. Given the output from the parsing example above, interpret() would construct a lambda and invoke it with the argument "Lisp". The lambda invocation would return "Lisp", which would be the output of the whole program.

As well as the input to execute, interpret() receives an execution context. This is the place where variables and their values are stored. When a piece of Lisp code is executed by interpret(), the execution context contains the variables that are accessible to that code.

These variables are stored in a hierarchy. Variables in the current scope are at the bottom of the hierarchy. Variables in the enclosing scope are in the level above. Variables in the scope enclosing the enclosing scope are in the level above that. And so on. For example, in the following code:

((lambda (a)
  ((lambda (b)
    (b a))
   "b"))
 "a")

On line 3, the execution context has two active scopes. The inner lambda forms the current scope. The outer lambda forms an enclosing scope. The current scope has b bound to "b". The enclosing scope has a bound to "a". When line 3 runs, the interpreter tries to look up b in the context. It checks the current scope, finds b and returns its value. Still on line 3, the interpreter tries to look up a. It checks the current scope and does not find a, so it tries the enclosing scope. There, it finds a and returns its value.

In Little Lisp, the execution context is modeled with an object made by calling the Context constructor. This takes scope, an object that contains variables and their values in the current scope. And it takes parent. If parent is undefined, the scope is the top, or global scope.

1   var Context = function(scope, parent) {
2     this.scope = scope;
3     this.parent = parent;
4
5     this.get = function(identifier) {
6       if (identifier in this.scope) {
7         return this.scope[identifier];
8       } else if (this.parent !== undefined) {
9         return this.parent.get(identifier);
10      }
11    };
12  };

We have seen how ((lambda (x) x) "Lisp") gets parsed. Let us see how the parsed code gets executed.

1   var interpret = function(input, context) {
2     if (context === undefined) {
3       return interpret(input, new Context(library));
4     } else if (input instanceof Array) {
5       return interpretList(input, context);
6     } else if (input.type === "identifier") {
7       return context.get(input.value);
8     } else {
9       return input.value;
10    }
11  };

The first time interpret() is called, context is undefined. Lines 2-3 are run to make an execution context.

When the initial context is instantiated, the constructor function takes the library object. This contains the functions built in to the language: first, rest and print. These functions are written in JavaScript.

interpret() recurses with the original input and the new context.

input contains the full example output from the parsing section:

[[{ type: 'identifier', value: 'lambda' }, [{ type: 'identifier', value: 'x' }],
  { type: 'identifier', value: 'x' }],
 { type: 'literal', value: 'Lisp' }]

Because input is an array and context is defined, lines 4-5 are run and interpretList() is called.

1   var interpretList = function(input, context) {
2     if (input.length > 0 && input[0].value in special) {
3       return special[input[0].value](input, context);
4     } else {
5       var list = input.map(function(x) { return interpret(x, context); });
6       if (list[0] instanceof Function) {
7         return list[0].apply(undefined, list.slice(1));
8       } else {
9         return list;
10      }
11    }
12  };

In interpretList(), line 5 maps over the input array and calls interpret() on each element. When interpret() is called on the lambda definition, interpretList() gets called again. This time, the input argument to interpretList() is:

[{ type: 'identifier', value: 'lambda' }, [{ type: 'identifier', value: 'x' }],
  { type: 'identifier', value: 'x' }]

Line 3 of interpretList() gets called, because lambda, the first element in the array, is a special form. special.lambda() is called to create the lambda function.

1   var special = {
2     lambda: function(input, context) {
3       return function() {
4         var lambdaArguments = arguments;
5         var lambdaScope = input[1].reduce(function(acc, x, i) {
6           acc[x.value] = lambdaArguments[i];
7           return acc;
8         }, {});
9
10        return interpret(input[2], new Context(lambdaScope, context));
11      };
12    }
13  };

special.lambda() takes the part of the input that defines the lambda. It returns a function that, when invoked, invokes the lambda on some arguments.

Line 3 begins the definition of the lambda invocation function. Line 4 stores the arguments passed to the lambda invocation. Line 5 starts creating a new scope for the lambda’s invocation. It reduces over the part of the input that defines the parameters of the lambda: [{ type: 'identifier', value: 'x' }]. It adds a key/value pair to the lambda scope for each lambda parameter in input and argument passed to the lambda. Line 10 invokes the lambda by calling interpret() on the lambda body: { type: 'identifier', value: 'x' }. It passes in the lambda context that contains the lambda’s scope and the parent context.

The lambda is now represented by the function returned by special.lambda().

interpretList() continues mapping over the input array by calling interpret() on the second element of the list: the "Lisp" string.

1   var interpret = function(input, context) {
2     if (context === undefined) {
3       return interpret(input, new Context(library));
4     } else if (input instanceof Array) {
5       return interpretList(input, context);
6     } else if (input.type === "identifier") {
7       return context.get(input.value);
8     } else {
9       return input.value;
10    }
11  };

This runs line 9 of interpret() which just returns the value attribute of the literal object: 'Lisp'. The map operation on line 5 of interpretList() is complete. list is:

[function(args) { /* code to invoke lambda */ },
 'Lisp']

Line 6 of interpretList() runs and finds that the first element of list is a JavaScript function. This means that the list is an invocation. Line 7 runs and invokes the lambda, passing the rest of list as arguments.

1   var interpretList = function(input, context) {
2     if (input.length > 0 && input[0].value in special) {
3       return special[input[0].value](input, context);
4     } else {
5       var list = input.map(function(x) { return interpret(x, context); });
6       if (list[0] instanceof Function) {
7         return list[0].apply(undefined, list.slice(1));
8       } else {
9         return list;
10      }
11    }
12  };

In the lambda invocation function, line 8 calls interpret() on the lambda body, { type: 'identifier', value: 'x' }.

1   function() {
2     var lambdaArguments = arguments;
3     var lambdaScope = input[1].reduce(function(acc, x, i) {
4       acc[x.value] = lambdaArguments[i];
5       return acc;
6     }, {});
7
8     return interpret(input[2], new Context(lambdaScope, context));
9   };

Line 6 of interpret() finds that input is an identifier atom. Line 7 looks up the identifier, x, in context and returns 'Lisp'.

1   var interpret = function(input, context) {
2     if (context === undefined) {
3       return interpret(input, new Context(library));
4     } else if (input instanceof Array) {
5       return interpretList(input, context);
6     } else if (input.type === "identifier") {
7       return context.get(input.value);
8     } else {
9       return input.value;
10    }
11  };

'Lisp' is returned by the lambda invocation function, which is returned by interpretList(), which is returned by interpret(), and that’s it.

Go to the GitHub repository to see all the code. And look at lis.py, the dazzlingly simple Scheme interpreter that Peter Norvig wrote in Python.

Mary rose cook 150
Tweet

What we look for in students

Update 2/25/14: Since we wrote this blog post, we’ve begun to deemphasize “student” in favor of “Hacker Schooler” and other context dependent words and phrases (see our blog post, “Words are hard”). While this post is out of date in its use of “student,” the rest of its advice is still relevant.

One of the surprising things we’ve learned while running Hacker School is that many of our students almost didn’t apply because they thought they wouldn’t get in. This includes people who are exceptional Hacker Schoolers, and who are exactly the type of people we look for.

This scares us, since it suggests there are people who know about Hacker School and would be a great fit, but who aren’t even applying because they don’t think they’re qualified.

The goal of this post is to be as transparent as possible about what we look for, so that you apply if you’re one of these people. Similarly, we want to make it as easy as possible for you to show us that you’re one of these people, so we don’t make a mistake and miss you when reviewing applications.1

Here are the qualities we look for, and some tips on how to convey them in your application and interviews:

  • You enjoy programming. We spend most of our time at Hacker School programming, or doing things directly related to programming (like code reviews), and we reject the carrots and sticks approach of traditional schooling. This means that you need to enjoy programming to be happy and do well at Hacker School. (This does not mean you need to be single-mindedly obsessed with programming. See our FAQ and this blog post for more.)

Admissions tips: Link to code you’ve written, a technical blog, or anything else that shows us that you program.2

  • You want to get significantly better. This applies regardless of if you’ve been programming for two months or two decades. We don’t think there’s a meaningful upper-bound for how good a programmer you can be. And even if there is, 99.9% of us aren’t close to it.

Admissions tips: Tell us about your weaknesses as a programmer, and how you’d like to grow. (If you’re a new programmer, this can be hard, since you might not even know what you don’t know. That’s fine! It’s ok if your answer is “I’ve only been programming for eight weeks, and I’ve never written a program from scratch that’s longer than 20 lines.”).

  • You’re friendly. We’re a tight-knit community, and we spend three months in close quarters. Purely for our own happiness we don’t want to admit jerks. We also think this is pedagogically important.

  • You’re self-directed. Hacker School isn’t a programming “bootcamp.” We don’t have a fixed curriculum that we force-feed you, nor will we dictate how you spend your time. There are a ton of resources to take advantage of here (from facilitators to residents to the other students), but it’s ultimately up to you to get the most out of Hacker School, and this requires that you’re able to organize and prioritize your own time.

  • You’re intellectually curious. Your curiosity is your north star at Hacker School, and we want to see that you get excited about ideas.

Admissions tips: Tell us about something that engaged you intellectually when answering the question about the most fascinating thing you’ve learned recently. We want to hear about something surprising you’ve learned, and it doesn’t need to be about programming. For example, it could be about high-frequency trading or how ancient Greek math was more practical than most people think.

  • You have a demonstrated capacity for rigor. We want to see that you know what it means to understand something deeply. Equally or perhaps more importantly, we also want to see that you’re aware when you don’t understand something.

Admissions tips: We care more about depth than breadth – we’d rather people have a deep understanding of one language than a superficial understanding of three. We want to see that you understand the code that you’ve written and don’t just bang things together until they work. If you’re really new to programming, we want to see that you’ve previously been successful in learning something hard, whether that’s law or biomedical engineering.

  • You’re introspective. Being introspective will help you make the most of your time at Hacker School. We think introspection and reflection are essential to growth: You need to be able to occasionally step back and assess your understanding and progress.

Admissions tips: Use the Hacker School admissions process as an opportunity to evaluate us and decide if Hacker School is a good choice for you. We love it when people ask thoughtful questions that show they’re engaged and actively thinking about if Hacker School is the right choice for them.

If you think Hacker School is a good fit for you, we’d love to see you apply, and we hope this post helps you with your application.

  1. Some basic tips for applying: Know what Hacker School is and what it’s not (you should read our about page, FAQ, and User’s Manual beforehand); use proper spelling and grammar (you don’t need to be “formal,” but you should write in full sentences); and demonstrate good judgement in how you present yourself in your application.

  2. One negative signal for us is if you previously programmed and then stopped (e.g., you studied CS and then became a product manager). If this applies to you, tell us why you left programming and what’s changed in the interim. We also want to see signs that you’ve already started coding again.

    Update: A couple of people were confused by this point, so to clarify: We’ve had students who went through traditional CS programs, hated them (and were almost driven away from programming), and were awesome Hacker Schoolers. So the above is a negative signal, but it only matters absent stronger positive signals. E.g., if the person actually programs, that’s a positive signal, and counts much more for us.

One more resident: Alex Payne

We have one more resident to announce for the summer 2013 batch: Alex Payne. Alex co-authored O'Reilly’s Programming Scala, was one of the first programmers at Twitter, and most recently served for two years as CTO of Simple.

Alex was in our first group of residents in the fall batch last year. He embodies what we look for in residents: he’s talented, accomplished, and knowledgeable, but also humble, unassuming, and incredibly friendly. We’re delighted to welcome him back this summer.

Allison kaptur 150
Tweet

Publishing the Hacker School User’s Manual

We want to be more transparent about Hacker School. We think this will help the world understand Hacker School better and make it easier for us to attract the kind of friendly, ambitious, and rigorous students who excel here.

As a first step towards more transparency, we’ve decided to publish the Hacker School User’s Manual. This was written as an internal document to help new students navigate Hacker School, but we also think it’s a good way to help people understand what we’re about. Hopefully this will answer some of the more common questions about how Hacker School works and maybe even encourage a few more people to apply.

The document is virtually identical to the version we use internally, except for the “Other resources” section. This section has a bunch of URLs and email addresses that are only useful if you’re a Hacker Schooler. It has been replaced with a brief summary.

There are many parts of this document we’d like to rewrite. There are sections that are out of date or no longer accurately reflect our opinions, but we don’t want to let the perfect be the enemy of the good. The information in the User’s Manual is much better than anything else we have published so far, and if it helps a few more people better understand Hacker School, it’s worth it.

There’s a lot more to do in our quest to be more transparent – there are blog posts to write, copy to revise, and of course the User’s Manual has to be updated before the next batch starts – but we think this release is a good start. Happy reading!

David albert 150
Tweet

Peter Norvig and eight others are Hacker School residents

We’re thrilled to announce nine residents for the summer 2013 batch of Hacker School.

Joining us for the first time will be Nada Amin, Mel Chua, Lindsey Kuper, Kevin Lynagh, Peter Norvig, Brandon Rhodes, and John Myles White. Two of our favorite people, Stefan Karpinski and Jessica McKellar, are each returning for their third residency.

They’re all smart, thoughtful, accomplished, and genuinely nice people, and we’re incredibly excited to have each of them. Check out our residents page to learn more about them.

We want to make Hacker School the best place on earth to become a great programmer, and we want Hacker School to be the most productive three months of our students' lives.

That’s why we started our residents program last year to bring the best programmers we can find to Hacker School. Residents come for one or two weeks and work directly with students: They pair program, do code reviews, give short talks, run seminars, and bond with the batch.

If you’re a type systems nerd; if you want to dig your teeth into Scala, Clojure, Rust, or Julia; if you want to learn Python networking with the folks who wrote the books; or if you just want to pair program with Peter Norvig, Hacker School is for you.

If you want to become a much better programmer and you’re intimidated by this list of residents, Hacker School is for you. (Read Nick’s blog post, What we mean when we say “hacker”, if you’re not sure if we mean you.)

Can I still apply?

Yes! The application deadline is this Thursday, May 2nd. After that, we’ll still accept applications, but it’s much harder to get admitted late. Apply now.

[Edit May 24th: And Alex Payne too!]

Allison kaptur 150
Tweet

New residents and sponsors

Today, I’m going to share some things we learned from the last batch of Hacker School, and announce our winter 2013 residents and grant sponsors.

While it’s too late to apply for our current batch, applications are open for our summer 2013 batch.

Reflections on the residency program

Last batch, as an experiment, we invited a few great programmers we respect tremendously to be residents at Hacker School, each for one or two weeks. The experiment was a huge success and Stefan Karpinski, Jessica McKellar, David Nolen, Alex Payne, and Peter Seibel deserve all the credit. Hacker Schoolers got to talk and pair with them, learn from their experiences, and become friends with them.

Now that we have some distance between ourselves and the last batch, I want to share some of the most interesting things we learned from our residency program.

We were lucky to be able to pepper our residents with questions about how they learn, and there was a surprising amount of variety to their answers. Some of our residents said that they learned best with self-directed study, while others said that they need more structure. One even said he learns best with the constraints of a job.

This has affected how we run Hacker School in two ways. First, it has inspired us to experiment with adding more optional structure to the batch, and second, it has made us much more wary of the idea that there is one true way to learn programming. If Hacker School is going to be the best place in the world to become a better programmer, we must be able to cater to all types of students.

Next, having residents at Hacker School reaffirmed our belief that reflection and introspection are integral to the learning process. Despite the differences in how they approached learning, all of our residents have thought a lot about the paths they took to get to where they are now. Though we recently removed a daily reflection feature from our internal website because it wasn’t getting used enough, introspection has become a core part of the way programming is learned at Hacker School1.

Lastly, there’s a lot of value in watching experienced programmers work. Two of our most successful group activities have been seminars watching Peter do a live refactoring of a small tic-tac-toe program and having David walk through and explain an implementation of miniKanren. Watching great programmers work shows us how much we can improve and inspires us to try to get there.

New residents

Given how much their presence improved Hacker School, it was a no-brainer to invite all of our past residents back for another batch. We set our expectations appropriately given that, unlike us, they have concerns outside of Hacker School, and we were ecstatic to learn that Stefan, Jessica, and David were all able to come back for this batch. Residents are now a core part of Hacker School and we’re grateful that Stefan, Jessica, and David will be spending time with us again.

In addition, we’re excited to have two new residents joining us this batch: Jacob Kaplan-Moss and Glyph. Jacob is one of the creators of Django and Glyph is the creator of Twisted. In addition to their obvious technical qualifications, Jacob and Glyph are warm, friendly, and thoughtful and will help make Hacker School a safe place to ask questions and grow as a programmer. We’re looking forward to learning from their experience and we know they’ll help make Hacker School even better. You can read more about Jacob and Glyph on our residents page.

New sponsors for grants for women

Our goal is to make Hacker School the best place on earth to become a better programmer. One of the best predictors of how much you will improve during your time here is how comfortable you are admitting you don’t know something. The most successful Hacker Schoolers actively seek out things they don’t understand and fearlessly ask questions until they do. We want to create an environment where everyone, both men and women, are able to do these things and excel.

Women have made up a small minority of programmers for many years. When you are in a minority, it is easy to feel like you don’t belong and to be susceptible to stereotype threat. If you don’t feel like you belong at Hacker School, you won’t be as comfortable asking questions and admitting gaps in your knowledge, and if you can’t do that, you won’t improve as much as you could. This is one of the reasons we care so much about having gender parity at Hacker School. Our goal is to eliminate the negative effects of being in a minority that female programmers have to deal with in most of the rest of the world. While we’re not there yet (our past three batches have been 35-45% female), we’re working on fixing this problem by providing need-based grants to any female student who needs one2. Hacker School remains free to attend, but being in New York for three months without a job is not free and having cash on hand to cover the cost of living can make all the difference.

For the past three batches, generous and committed companies have stepped up to help us provide these grants that we cannot afford to give ourselves. We want to take a moment to thank the companies who are sponsoring grants for this batch, and the people at each one who made it possible. Thanks to Albert Ni at Dropbox, Marc Hedlund at Etsy, Scott Chacon at GitHub, and Marc Yun at PhotoShelter. All these companies care tremendously about supporting female programmers and are putting their money where their mouths are. Without these wonderful sponsors, Hacker School would not be possible. We are honored to have their support. Thank you!

Curious about Hacker School? Read more about us and apply for our summer batch.

  1. More on this in a future blog post.

  2. The grants have successfully boosted the number of women able to do Hacker School, without impacting men’s chances of being accepted, since we’ve been able to admit all of the qualified students who have applied to our recent batches. If you’d like to sponsor need-based grants for women or men to do Hacker School, please email us.

David albert 150
Tweet

Mary Rose Cook is a Hacker Schooler

Mary Rose Cook

Mary

We’re thrilled to announce that Mary Rose Cook is joining the Hacker School faculty!

Mary was in the summer 2012 batch, and we’ve been working on hiring her ever since.

Mary is the author of some really awesome open source projects, including Isla, a programming language for young children. She’s written multiple libraries, including Andro.js, Machine.js and Glazz. She’s also made two very fun and challenging games, most recently the inventive platformer, Empty Black.

I could go on raving about Mary’s many accomplishments, but I don’t think they capture why we’re so excited to have her joining us. Here’s the real reason we’re so happy: In addition to being a wonderful programmer, Mary is a delightful human being. She’s fun, funny, inquisitive, patient, friendly, creative and a pleasure to work with.

Please join us in welcoming Mary to Hacker School. Together, we’re going to continue building Hacker School into the best place on earth to learn and grow as a programmer.

— Hacker School

  • Alan
  • Allison
  • Dave
  • Mary
  • Nick
  • Sonali
  • Tom
  • Zach

(Alan, Allison, Dave, Mary, Nick, Sonali, Tom, and Zach)

There’s no magic: virtualenv edition

The more programming I do, the more often I find myself thinking, “Ah, that’s not magic.” I had one of these moments recently when dealing with a python virtual environment created by virtualenv. Virtualenv creates a sandboxed python environment with its own installation directories, separate from the system python and other virtual environments on your machine. This makes it a great way to test on multiple versions of python or to explore a new package that could break other things you care about.

How does it work? Well, the ‘magic’ works like this:
- create a virtual environment with virtualenv my_env
- chant the magic incantation source bin/activate
- watch as your previously-failing installation of pygame goes smoothly!
- To “turn off” your virtualenv, the magic incantation is deactivate.

Of course, it’s not actually magic. Virtualenv is a fairly simple (though clever) bash script that does only a couple of things. You don’t have to understand much bash scripting to see what’s going on. In fact, if you only know python, I’ll teach you all the bash you need to understand virtualenv right now.

I’m going to skip the actual creation of a virtual environment, and just focus on what happens when you activate and deactivate that environment. If you’d like to play along, pip install virtualenv, create a new virtual environment with virtualenv testenv and then cd into the testenv/ directory that was created. (Don’t run source bin/activate just yet.)

First, let’s look at that incantation, source bin/activate. What’s going on here? source is a bash command that runs a file, the same way you’d use import to run your python module.1 bin/activate is the bash script being run.

One other detail of source will be important. source runs the file provided in your current shell, not in a subshell. Thus it keeps the variables it creates or modifies around after the file is done executing. Since (almost) all that virtualenv does is modify environmental variables, this matters.

OK, now let’s look at bin/activate. Fire up the activate file in your favorite text editor.
The first thing to notice is that it’s only ~80 lines! Cool - we can handle this.

(The activate script is generated automatically by the virtualenv installation, and has some system-specific parameters, so your copy may be slightly different that mine. Mine looks exactly like this.)

The first thing we find is a comment:

# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly

We already know why this is true - it’s because of the behavior of source that we just learned. Running a bash file directly (e.g. calling activate from bin/) runs the script in a subshell - not what we want.

Onward:

deactivate () {
...
}

This is just a bash function definition. Function calls work just like commands in bash. Now we know that the commands included in this block are what runs when we say deactivate in our virtual environment.

The meat of the activate file is in lines 42 - 47:

# unset irrelevant variables
deactivate nondestructive

VIRTUAL_ENV="/Users/afk/examples/testenv"
export VIRTUAL_ENV

_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH

Starting with a call to deactivate ensures that any existing virtual environment is deactivated before a new one is created. Virtual environments are separate from each other; they can’t be nested.

The rest of this is pretty straightforward. export is the only other bash command we need to know, and it’s really simple: it just exports a variable into your current environment. It also ensures that environmental variables in processes spawned from the current one get the same values. Since we’re running the file via source, the effect is to set variables and then keep them after the activate script finishes running.

So the activate script does three primary things:
1. Sets a VIRTUAL_ENV bash environmental variable containing the virtual environment directory
2. Prepends that directory to your PATH
3. Sets the new PATH.

What is PATH? The PATH is an environmental variable representing a list of directories. Your system will look for programs and scripts in the order that directories are listed. The list is separated by colons.

Let’s see this in action. To see what your PATH looks like before you run the activate file, hop into your terminal and type echo $PATH. This prints out the value of PATH to the terminal. Mine looks in part like this (I’ve inserted line breaks for clarity):

/usr/local/bin:
/usr/local/sbin:
/usr/bin

All this says is that when I type a command like python, my system looks first in /usr/local/bin for python. If it can’t find it, it moves on to /usr/local/sbin, then to /usr/bin, and so on.

Now let’s run the activate file and see what changed. (Again, I’ve inserted line breaks for clarity.)

testenv\ $ source bin/activate
(testenv)testenv\ $ echo $PATH
/Users/afk/examples/testenv/bin:
/usr/local/bin:
/usr/local/sbin:
/usr/bin

Sure enough, that testenv directory has been prepended to my PATH. Now bash will look for python, or any other system command, first in the bin/ directory here in my testenv. What’s in there? Let’s take a look:

testenv\ $ ls bin/
activate            easy_install        python
activate.csh        easy_install-2.7    python2
activate.fish       pip                 python2.7
activate_this.py    pip-2.7

There’s our activate file that we’ve been examining, plus a version of python! So this is the python installation that will get modified if we install packages, and the python that will be run by python. For easy confirmation of this, we can use which:

(testenv)testenv\ $ which python
/Users/afk/examples/testenv/bin/python

We’re using the testenv python, not the system python (which is found in usr/bin/).

Notice that activate also modified my bash prompt (PS1). We’ll skip some of the details here - the important point is that this code stores your old PS1 and inserts the name of the virtualenv into the new one.2

We’re done with our virtualenv for now - let’s come back to deactivate.

deactivate () {
    unset pydoc

    # reset old environment variables
    if [ -n "$_OLD_VIRTUAL_PATH" ] ; then
        PATH="$_OLD_VIRTUAL_PATH"
        export PATH
        unset _OLD_VIRTUAL_PATH
    fi
    if [ -n "$_OLD_VIRTUAL_PYTHONHOME" ] ; then
        PYTHONHOME="$_OLD_VIRTUAL_PYTHONHOME"
        export PYTHONHOME
        unset _OLD_VIRTUAL_PYTHONHOME
    fi

    #[special case omitted for brevity]

    if [ -n "$_OLD_VIRTUAL_PS1" ] ; then
        PS1="$_OLD_VIRTUAL_PS1"
        export PS1
        unset _OLD_VIRTUAL_PS1
    fi

    unset VIRTUAL_ENV
    if [ ! "$1" = "nondestructive" ] ; then
    # Self destruct!
        unset -f deactivate
    fi
}

deactivate calls export to restore the old environmental variables, then calls unset to remove unneeded variables from the environment. (You can verify this from the terminal by using the command env to view all your environmental variables.) Finally, deactivate calls unset -f deactivate to remove the deactivate function itself. (-f removes a function.) The function is now gone from the environment, which you can easily verify:

(testenv)testenv\ $ deactivate
testenv\ $ deactivate
-bash: deactivate: command not found

Our PS1, PATH, and PYTHONHOME end up with their original values.

There you have it - no magic, and just a tiny bit of bash scripting to understand the power of a virtual environment.

  1. source is the same as the dot operator .

  2. There’s a particularly bewildering bit of code in these lines:

    if [ "x" != x ] ; then
      PS1="$PS1"

    The if statement as written will always return false - but the following line doesn’t do anything anyway. This turns out to be a consequence of the system-dependent nature of virtualenv and the fact that the activate script is automatically generated. See here for a more detailed explanation.

Allison kaptur 150
Tweet

List comprehensions in eight lines of Clojure

List comprehensions are a syntactic construct that allow you to create new lists in an elegant and concise way.

Here’s a question, though: If your language did not support list comprehensions, what would it take to add them? For a language like Ruby, you may have to dig into the language implementation itself. If you’re using a Lisp, you can write a macro.

Lisp macros allows you to transform your code at compile time. Macro calls are run before the rest of your code during a process called macro expansion. You define macros almost like any other function, except that the arguments to a macro are unevaluated Lisp code. Because Lisp code is just data, you can operate on it like any other data.

This is how you can implement list comprehensions in eight lines of Clojure.

(defmacro list-comp [[binding seq-expr & bindings] body-expr]
  (cond (not binding)
        `(list ~body-expr)

        (= :when binding)
        `(when ~seq-expr (list-comp ~bindings ~body-expr))

        :else
        `(mapcat (fn [~binding] (list-comp ~bindings ~body-expr))
                 ~seq-expr)))

I’ve been thinking about this macro for three days now, and it’s still terribly exciting to me. It’s the macro I wish I had seen when I was learning about them. It takes full advantage of Clojure at compile-time and uses both higher-order functions and recursion.

Let’s see what it can do.

;; copy lists
(list-comp [x (range 10)]
  x)
;;=> (0 1 2 3 4 5 6 7 8 9)

;; double every number
(list-comp [x (range 5)]
  (* x 2))
;;=> (0 2 4 6 8)

;; filter out elements
(list-comp [x (range 10) :when (odd? x)]
   x)
;;=> (1 3 5 7 9)

;; nested bindings
(list-comp [x "abc"
            y [0 1 2]]
  [x y])
;;=> ([\a 0] [\a 1] [\a 2] [\b 0] [\b 1] [\b 2] [\c 0] [\c 1] [\c 2])

;; list permutations
(defn permutations [xs]
  (if-not (seq xs)
    (list ())
    (list-comp [x xs
                ys (permutations (list-comp [z xs :when (not= z x)] 
                                   z))]
      (conj ys x))))

(permutations [1 2 3])
;;=> ((1 2 3) (1 3 2) (2 1 3) (2 3 1) (3 1 2) (3 2 1))

At compile time, it is transformed into code that produces the correct result. It’s a pretty handy macro. We can see how it works by looking at some macro expansions.

At its core, list-comp is just a recursive transformation. This is made obvious by looking at the first macroexpansion of a comprehension with nested bindings.

(macroexpand '(list-comp [x "ab" y [0 1]] [x y]))

;; yields

(mapcat (fn [x] (list-comp [y [0 1]] [x y]))
        "ab")
;;=> ([\a 0] [\a 1] [\b 0] [\b 1])

If we simply assume that the inner call to list-comp does the right thing – that is, it successfully binds y to 0 and then 1 – then it is clear that this expansion works. The call to mapcat will take care of binding x properly, and then concatenate the results. This will work with an arbitrary level of nesting.

What happens when we use the :when clause?

(macroexpand '(list-comp [x "ab" :when (= x \a) y [0 1]] [x y]))

;; yields

(mapcat (fn [x]
          (when (= x \a)
            (list-comp (y [0 1]) [x y])))
        "ab")
;;=> ([\a 0] [\a 1])

If the condition is truthy, we execute the inner comprehension. Otherwise, when returns nil, which, when concatenated with some other list, returns that other list.

However, the skeptical reader just noticed a possible wrench in the works. If :when wraps the recursive call, what happens when there is no recursive call left? That is, doesn’t everything break if :when is last form to appear in the bindings?

Thankfully, it doesn’t. The base-case is not the case of a single binding pair, but instead the case of an empty set of bindings. Then, we just return the body’s result as a singleton list.

(macroexpand '(list-comp [] :foo))

;; yields

(list :foo)

This means that our :when clause can appear at the end of the bindings vector.

(clojure.walk/macroexpand-all
  '(list-comp [x (range 10) :when (even? x)] x))

;; yields

(mapcat (fn [x]
          (when (even? x)
            (list x))) ;; the empty-binding case
        (range 10))
;;=> (0 2 4 6 8)

And that’s all there is to it.

A special thanks to Alan O'Donnell and Darius Bacon for joining me on the quest for the simplest list comprehension macro.

Edit Jan 2, 2013: I should have mentioned this, but for posterity’s sake, Clojure comes with list comprehensions already built in. See Clojure’s for macro.

Zach allaun 150
Tweet

What we mean when we say “hacker”

What do you think of when you hear the word “hacker?” If you’re like many people, you think of a 20-something white guy hunched over his computer, intently working alone at 4am, surrounded by darkness and empty soda cans.

That’s the popular culture view of a hacker, but that’s not at all what we want in potential Hacker Schoolers. Here’s what we mean by the word “hacker”:

Someone who is intellectually curious. Hackers enjoy tinkering and understanding how things work. They revel in going deeper into things, and love asking questions: How can I make my code easier to debug? What is making my program slow? How does this work?

Someone who enjoys programming. We spend most of our time at Hacker School programming, there aren’t grades or tests to make you study, and you don’t get a degree or certificate for doing Hacker School, so it’s important that you like coding.

We used to say applicants must “love” programming, but we’ve learned that was a mistake. While it sounds good, it doesn’t actually describe what we care about, and it was dissuading qualified people from applying.

In fact, many of our best students have said they almost didn’t apply because they worried they didn’t love programming enough. Hacker School alum David Peter expressed this fear perfectly:

One of the questions in the interview was, “Do you love programming?” I said yes, because I loved it more than most people I knew. But was it love? I enjoyed writing and drawing equally, if not more. After Hacker School, I’m revisiting these hobbies. Programming wasn’t my first love.“

(You should read David’s entire post if you haven’t already. It’s moving.)

In reality, we just want people who enjoy programming, and who are serious about improving in a focused way. You should like programming enough that the prospect of spending three months getting better at it sounds like fun and not a chore.

If you like feeling your brain stretch as you understand a new concept, or take pride in making your code more concise and elegant, or delight in building things, you’re probably a good fit for Hacker School.

Someone who enjoys sharing what she learns with others. The most respected hackers are those who have built or discovered great things and shared them with the world. At Hacker School, everyone writes free and open source software, and everyone both learns from and teaches everyone else.

Someone who is friendly and communicates well. This is especially important for Hacker School, since we spend three months working closely together. If you can’t communicate effectively, it’s both hard to ask for help and to help others.

Now here’s what we don’t care about

You don’t need to have a lot of programming experience. While not every programmer is in a place where Hacker School makes sense, you don’t need to have started your own open source projects or mastered two programming languages to be an awesome Hacker Schooler. If you’re at the point where you can write simple programs from scratch, you’ve probably got enough experience.

The lower bound for experience for students who have successfully done Hacker School seems to be about two months. That is, we’ve had exceptional students who have come to Hacker School with as little as two months of programming experience and who have done enormously well. We don’t think there’s an upper bound for experience.

You don’t need to only care about programming. If you thought Hacker School wasn’t for you because you don’t spend all your time programming, don’t worry. While all Hacker Schoolers enjoy programming, we also like lots of other things, from ballroom dancing to dumpling tours. You don’t need to be single-mindedly obsessed with programming to excel at Hacker School.

You don’t need to have the superficial trappings commonly associated with hackers. None of that stuff matters at Hacker School. The factors that determine if people are good Hacker Schoolers have nothing to do with how they look or other superficial attributes, and everything to do with being a thoughtful, curious person who enjoys programming and wants to focus on improving.

If you enjoy programming and want to spend three, focused months getting better in a diverse and supportive environment, we want you at Hacker School.

Exploring Emacs

Here’s a fun idea in case you want to practice reading code and using Emacs: read Emacs in Emacs!

Emacs makes it easy to see the code behind its commands. For instance, if you want to see how a particular command-key sequence works, type C-h k and then the key sequence into the modeline prompt.

If you’re curious how C-h k itself works, type C-h k C-h k. According to the *Help* buffer that appears, C-h k corresponds to the Emacs Lisp function describe-key defined in help.el. Move your cursor onto the help.el text in the Help buffer and press enter. You’re now looking at the Emacs Lisp source for describe-key.

If you’d like to see how a particular Emacs Lisp function or special form works, type C-h f and then the name of the function or special form.

Perhaps you’re curious how lambda works: type C-h f lambda and then hit enter. Another *Help* buffer will appear, saying that “lambda is a Lisp macro in subr.el”. As before, place your cursor inside subr.el and then press enter; Emacs will transport you right to the definition of lambda.

Emacs isn’t Emacs Lisp all the way down, though. For instance, addition is presumably defined in C. C-f + confirms this, noting that “+ is a built-in function defined in data.c”. If you move your cursor over data.c and press enter, Emacs will probably prompt you at the modeline for the location of your Emacs C source directory; it’s likely that when you installed Emacs, you only downloaded pre-compiled binaries. You can find source corresponding to your version of Emacs here. Once you tell Emacs where to look, you can hop into Emacs’s C source code, too.

It’s fun to guess whether a particular function or special form is defined in Emacs Lisp or C. The answer can also change between versions of Emacs. For example, until quite recently, the special form for defining a new Emacs Lisp function, defun, was defined in C. If you’re unfamiliar with Lisp, you might wonder how it could be otherwise! But according to my *Help* buffer in Emacs 24.3.50.1, “defun is a Lisp macro defined in byte-run.el”.

The exuberant comment above defun’s definition in byte-run.el is worth reinstalling Emacs from trunk: “Now that we defined defmacro we can use it!”. Bootstrapping defun is now the very first use of defmacro in all of Emacs. If you’re curious where defmacro comes from, you know what to do :)

Stop waiting: Winter 2013 applications are open

tl;dr: We’ve opened applications for our winter 2013 batch. This will be our sixth batch, and it will run from February 11 to May 2.

The deadline for applications is Tuesday, January 1st at 11:59pm ET. We’ll begin sending out invitations for interviews then, and we’ll schedule Skype interviews for the first two weeks of January. We’ll consider applications submitted after the deadline, but your odds will be lower.

Stop waiting

If you’re like me, you read a lot of blog posts about self-improvement: Posts about how you can be more productive, or understand hard concepts, or become a better programmer.

The internet is full of amazing information. In fact, there’s already more technical knowledge freely available online than any single person could possibly learn in a lifetime.

The real answer for how to get better is to sit down and work at it. It takes time, concentration, and effort.

It also requires breaking out of our routines and bad habits. Simply resolving to try harder this year isn’t going to do much in the long-run. Trying the same thing repeatedly and expecting different results is the very definition of insanity.

That’s why we started Hacker School. We wanted to become better programmers, and we knew that meant time, concentration and hard work. And it meant breaking our old bad habits of reading Hacker News and blog posts about how other people were becoming better programmers.

Hacker School isn’t a shortcut. You won’t become exceptional overnight. But the truth is nobody becomes exceptional overnight. And nobody becomes exceptional reading blog posts on maximizing their efficiency. All the shortcuts you read online are really detours; they just run out the clock on your life without getting you any closer to where you want to be.

Doing Hacker School structures your life such that the default expectation is for you to spend the majority of your time programming. That’s it. The group sits in a room together eight hours a day, four days a week for twelve weeks. We check-in with each other about what we’ve built and learned, what we’re doing next, and where we’re stuck. We explain the things we understand to others in the group who don’t yet understand them, and they explain the things they understand to us.

We do this without judgement or snide remarks. We do this because people’s fear of appearing stupid is a serious impediment to becoming better, and the goal of Hacker School is to remove as many obstacles in the way of growth as possible.

Hacker School is not a magic elixir that will automatically make you an exceptional programmer. But it does remove distractions, and give you dedicated time to focus. And apply mild social pressure. And surround you with supportive, friendly people working towards similar goals. And give you direct access to people with more experience and knowledge.

Reading blog posts will no sooner make you a better programmer than reading fitness magazines will give you great abs. If you want to be buff you have to go to the gym. If you want to be a great programmer you have to write code.

Come to Hacker School.

We’ll have more to announce soon, but we wanted to give people as much time as possible to apply. If you have any questions, please check our FAQ. If they’re not answered there, please email admissions@hackerschool.com.

We’re thankful for GitHub and Palantir

With Thanksgiving nearly upon us, we want to take a moment to thank some people who have been instrumental to making Hacker School a reality.

But first, a brief story.

We opened applications for our fifth batch of Hacker School in the beginning of August. When we announced the batch, Etsy had already generously agreed to sponsor 10 grants for female programmers. We were also in talks with several other companies to provide additional scholarships, and we were confident that they would come through.

But as anyone who’s ever tried to make a business deal knows, nothing is final until it’s final, and things can fall apart at any moment. And that’s exactly what happened. A couple companies wanted to attach strings to their grants, like requiring that students intern for them or otherwise prioritize them after Hacker School. That didn’t fit with the spirit of Hacker School, and so those deals fell through. First one, then another.

By then, we were well into September. We were in the middle of sending out acceptances, and we were all out of grants. We had qualified women who were ready and excited to do Hacker School, but who couldn’t up and leave for New York without financial assistance. The batch was only a few weeks away, and they needed an answer.

Thankfully, that’s when GitHub and Palantir stepped up to provide additional grants. They were responsive, helpful, and most importantly just cared about helping more female programmers do Hacker School.

We want to personally thank Scott Chacon at GitHub and Ari Gesher at Palantir for coming through for us.

Thank you.

Alex Payne, Peter Seibel, Jessica McKellar, David Nolen, and Stefan Karpinski are Hacker School Residents

When we announced our residency program last month, we didn’t know what type of response we’d get. Would it be possible to get prominent progammers to take time away from their normal lives and jobs and spend a couple weeks at Hacker School?

We’re happy to say we now know the answer: Yes, it’s possible. And we’re even happier to announce our first set of Residents: Alex Payne, Peter Seibel, Jessica McKellar, David Nolen, and Stefan Karpinski. You can read all about them on our Residents page.

We’re thrilled to have such a wonderful group of Residents for our fall batch. They are all phenomenally smart and warm people, with significant expertise and passion, and are genuinely interested in helping others become better programmers.

If you’d like to hack with the creator of Julia, or write Scala with the guy who wrote the book on it, or start contributing to Twisted with its maintainer, or if you simply want to grow as a programmer in a friendly, supportive, and open environment, you should apply to Hacker School.

Understanding C by learning assembly

Last time, Alan showed how to use GDB as a tool to learn C. Today I want to go one step further and use GDB to help us understand assembly as well.

Abstraction layers are great tools for building things, but they can sometimes get in the way of learning. My goal in this post is to convince you that in order to rigorously understand C, we must also understand the assembly that our C compiler generates. I’ll do this by showing you how to disassemble and read a simple program with GDB, and then we’ll use GDB and our knowledge of assembly to understand how static local variables work in C.

Note: All the code in this post was compiled on an x86_64 CPU running Mac OS X 10.8.1 using Clang 4.0 with optimizations disabled (-O0).

Learning assembly with GDB

Let’s start by disassembling a program with GDB and learning how to read the output. Type the following program into a text file and save it as simple.c:

int main()
{
    int a = 5;
    int b = a + 6;
    return 0;
}

Now compile it with debugging symbols and no optimizations and then run GDB:1

$ CFLAGS="-g -O0" make simple
cc -g -O0    simple.c   -o simple
$ gdb simple

Inside GDB, we’ll break on main and run until we get to the return statement. We put the number 2 after next to specify that we want to run next twice:

(gdb) break main
(gdb) run
(gdb) next 2

Now let’s use the disassemble command to show the assembly instructions for the current function. You can also pass a function name to disassemble to specify a different function to examine.

(gdb) disassemble
Dump of assembler code for function main:
0x0000000100000f50 <main+0>:    push   %rbp
0x0000000100000f51 <main+1>:    mov    %rsp,%rbp
0x0000000100000f54 <main+4>:    mov    $0x0,%eax
0x0000000100000f59 <main+9>:    movl   $0x0,-0x4(%rbp)
0x0000000100000f60 <main+16>:   movl   $0x5,-0x8(%rbp)
0x0000000100000f67 <main+23>:   mov    -0x8(%rbp),%ecx
0x0000000100000f6a <main+26>:   add    $0x6,%ecx
0x0000000100000f70 <main+32>:   mov    %ecx,-0xc(%rbp)
0x0000000100000f73 <main+35>:   pop    %rbp
0x0000000100000f74 <main+36>:   retq   
End of assembler dump.

The disassemble command defaults to outputting instructions in AT&T syntax, which is the same syntax used by the GNU assembler.2 Instructions in AT&T syntax are of the format mnemonic source, destination. The mnemonic is a human readable name for the instruction. Source and destination are operands and can be immediate values, registers, memory addresses, or labels. Immediate values are constants, and are prefixed by a $. For instance, $0x5 represents the number 5 in hexadecimal. Register names are prefixed by a %.

Registers

It’s worth taking a quick detour to understand registers. Registers are data storage locations directly on the CPU. With some exceptions, the size, or width, of a CPU’s registers define its architecture. So if you have a 64-bit CPU, your registers will be 64 bits wide. The same is true of 32-bit CPUs (32-bit registers), 16-bit CPUs, and so on.3 Registers are very fast to access and are often the operands for arithmetic and logic operations.

The x86 family has a number of general and special purpose registers. General purpose registers can be used for any operation and their value has no particular meaning to the CPU. On the other hand, the CPU relies on special purpose registers for its own operation and the values stored in them have a specific meaning depending on the register. In our example above, %eax and %ecx are general purpose registers, while %rbp and %rsp are special purpose registers. %rbp is the base pointer, which points to the base of the current stack frame, and %rsp is the stack pointer, which points to the top of the current stack frame. %rbp always has a higher value than %rsp because the stack starts at a high memory address and grows downwards. If you are unfamiliar with the call stack, you can find a good introduction on Wikipedia.

One quirk of the x86 family is that it has maintained backwards compatibility all the way back to the 16-bit 8086 processor. As x86 moved from 16-bit to 32-bit to 64-bit, the registers were expanded and given new names so as to not break backwards compatibility with code that was written for older, narrower CPUs.

Take the general purpose register AX, which is 16 bits wide. The high byte can be accessed with the name AH, and the low byte with the name AL. When the 32-bit 80386 came out, the Extended AX register, or EAX, referred to the 32-bit register, while AX continued to refer to a 16-bit register that made up the lower half of EAX. Similarly, when the x86_64 architecture came out, the “R” prefix was used and EAX made up the lower half of the 64-bit RAX register. I’ve included a diagram below based on a Wikipedia article to help visualize the relationships I described:

|__64__|__56__|__48__|__40__|__32__|__24__|__16__|__8___|
|__________________________RAX__________________________|
|xxxxxxxxxxxxxxxxxxxxxxxxxxx|____________EAX____________|
|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|_____AX______|
|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|__AH__|__AL__|

Back to the code

This should be enough information to start reading our disassembled program.

0x0000000100000f50 <main+0>:    push   %rbp
0x0000000100000f51 <main+1>:    mov    %rsp,%rbp

The first two instructions are called the function prologue or preamble. First we push the old base pointer onto the stack to save it for later. Then we copy the value of the stack pointer to the base pointer. After this, %rbp points to the base of main’s stack frame.

0x0000000100000f54 <main+4>:    mov    $0x0,%eax

This instruction copies 0 into %eax. The x86 calling convention dictates that a function’s return value is stored in %eax, so the above instruction sets us up to return 0 at the end of our function.

0x0000000100000f59 <main+9>:    movl   $0x0,-0x4(%rbp)

Here we have something we haven’t encountered before: -0x4(%rbp). The parentheses let us know that this is a memory address. Here, %rbp is called the base register, and -0x4 is the displacement. This is equivalent to %rbp + -0x4. Because the stack grows downwards, subtracting 4 from the base of the current stack frame moves us into the current frame itself, where local variables are stored. This means that this instruction stores 0 at %rbp - 4. It took me a while to figure out what this line was for, but it seems that clang allocates a hidden local variable for an implicit return value from main.

You’ll also notice that the mnemonic has the suffix l. This signifies that the operands will be long (32 bits for integers). Other valid suffixes are byte, short, word, quad, and ten. If you see an instruction that does not have a suffix, the size of the operands are inferred from the size of the source or destination register. For instance, in the previous line, %eax is 32 bits wide, so the mov instruction is inferred to be movl.

0x0000000100000f60 <main+16>:   movl   $0x5,-0x8(%rbp)

Now we’re getting into the meat of our sample program! The first line of assembly is the first line of C in main and stores the number 5 in the next available local variable slot (%rbp - 0x8), 4 bytes down from our last local variable. That’s the location of a. We can use GDB to verify this:

(gdb) x &a
0x7fff5fbff768: 0x00000005
(gdb) x $rbp - 8
0x7fff5fbff768: 0x00000005

Note that the memory addresses are the same. You’ll notice that GDB sets up variables for our registers, but like all variables in GDB, we prefix it with a $ rather than the % used in AT&T assembly.

0x0000000100000f67 <main+23>:   mov    -0x8(%rbp),%ecx
0x0000000100000f6a <main+26>:   add    $0x6,%ecx
0x0000000100000f70 <main+32>:   mov    %ecx,-0xc(%rbp)

We then move a into %ecx, one of our general purpose registers, add 6 to it and store the result in %rbp - 0xc. This is the second line of C in main. You’ve maybe figured out that %rbp - 0xc is b, which we can verify in GDB:

(gdb) x &b
0x7fff5fbff764: 0x0000000b
(gdb) x $rbp - 0xc
0x7fff5fbff764: 0x0000000b

The rest of main is just cleanup, called the function epilogue:

0x0000000100000f73 <main+35>:   pop    %rbp
0x0000000100000f74 <main+36>:   retq   

We pop the old base pointer off the stack and store it back in %rbp and then retq jumps back to our return address, which is also stored in the stack frame.

So far we’ve used GDB to disassemble a short C program, gone over how to read AT&T assembly syntax, and covered registers and memory address operands. We’ve also used GDB to verify where our local variables are stored in relation to %rbp. Now we’re going to use our newly acquired skills to explain how static local variables work.

Understanding static local variables

Static local variables are a very cool feature of C. In a nutshell, they are local variables that only get initialized once and persist their values across multiple calls to the function where they are defined. A simple use case for static local variables is a Python-style generator. Here’s one that generates all of the natural numbers up to INT_MAX:

/* static.c */
#include <stdio.h>
int natural_generator()
{
        int a = 1;
        static int b = -1;
        b += 1;
        return a + b;
}

int main()
{
        printf("%d\n", natural_generator());
        printf("%d\n", natural_generator());
        printf("%d\n", natural_generator());

        return 0;
}

When compiled and run, this program prints the first three natural numbers:

$ CFLAGS="-g -O0" make static
cc -g -O0    static.c   -o static
$ ./static
1
2
3

But how does this work? To understand static locals, we’re going to jump into GDB and look at the assembly. I’ve removed the address information that GDB adds to the disassembly so that everything fits on screen:

$ gdb static
(gdb) break natural_generator
(gdb) run
(gdb) disassemble
Dump of assembler code for function natural_generator:
push   %rbp
mov    %rsp,%rbp
movl   $0x1,-0x4(%rbp)
mov    0x177(%rip),%eax        # 0x100001018 <natural_generator.b>
add    $0x1,%eax
mov    %eax,0x16c(%rip)        # 0x100001018 <natural_generator.b>
mov    -0x4(%rbp),%eax
add    0x163(%rip),%eax        # 0x100001018 <natural_generator.b>
pop    %rbp
retq   
End of assembler dump.

The first thing we need to do is figure out what instruction we’re on. We can do that by examining the instruction pointer or program counter. The instruction pointer is a register that stores the memory address of the next instruction. On x86_64, that register is %rip. We can access the instruction pointer using the $rip variable, or alternatively we can use the architecture independent $pc:

(gdb) x/i $pc
0x100000e94 <natural_generator+4>:  movl   $0x1,-0x4(%rbp)

The instruction pointer always contains the address of the next instruction to be run, which means the third instruction hasn’t been run yet, but is about to be.

Because knowing the next instruction is useful, we’re going to make GDB show us the next instruction every time the program stops. In GDB 7.0 or later, you can just run set disassemble-next-line on, which shows all the instructions that make up the next line of source, but we’re using Mac OS X, which only ships with GDB 6.3, so we’ll have to resort to the display command. display is like x, except it evaluates its expression every time our program stops:

(gdb) display/i $pc
1: x/i $pc  0x100000e94 <natural_generator+4>:  movl   $0x1,-0x4(%rbp)

Now GDB is set up to always show us the next instruction before showing its prompt.

We’re already past the function prologue, which we covered earlier, so we’ll start right at the third instruction. This corresponds to the first source line that assigns 1 to a. Instead of next, which moves to the next source line, we’ll use nexti, which moves to the next assembly instruction. Afterwards we’ll examine %rbp - 0x4 to verify our hypothesis that a is stored at %rbp - 0x4.

(gdb) nexti
7           b += 1;
1: x/i $pc  mov   0x177(%rip),%eax # 0x100001018 <natural_generator.b>
(gdb) x $rbp - 0x4
0x7fff5fbff78c: 0x00000001
(gdb) x &a
0x7fff5fbff78c: 0x00000001

They are the same, just as we expected. The next instruction is more interesting:

mov    0x177(%rip),%eax        # 0x100001018 <natural_generator.b>

This is where we’d expect to find the line static int b = -1;, but it looks substantially different than anything we’ve seen before. For one thing, there’s no reference to the stack frame where we’d normally expect to find local variables. There’s not even a -0x1! Instead, we have an instruction that loads 0x100001018, located somewhere after the instruction pointer, into %eax. GDB gives us a helpful comment with the result of the memory operand calculation and a hint telling us that natural_generator.b is stored at this address. Let’s run this instruction and figure out what’s going on:

(gdb) nexti
(gdb) p $rax
$3 = 4294967295
(gdb) p/x $rax
$5 = 0xffffffff

Even though the disassembly shows %eax as the destination, we print $rax, because GDB only sets up variables for full width registers.

In this situation, we need to remember that while variables have types that specify if they are signed or unsigned, registers don’t, so GDB is printing the value of %rax unsigned. Let’s try again, by casting %rax to a signed int:

(gdb) p (int)$rax
$11 = -1

It looks like we’ve found b. We can double check this by using the x command:

(gdb) x/d 0x100001018
0x100001018 <natural_generator.b>:  -1
(gdb) x/d &b
0x100001018 <natural_generator.b>:  -1

So not only is b stored at a low memory address outside of the stack, it’s also initialized to -1 before natural_generator is even called. In fact, even if you disassembled the entire program, you wouldn’t find any code that sets b to -1. This is because the value for b is hardcoded in a different section of the sample executable, and it’s loaded into memory along with all the machine code by the operating system’s loader when the process is launched.4

With this out of the way, things start to make more sense. After storing b in %eax, we move to the next line of source where we increment b. This corresponds to the next two instructions:

add    $0x1,%eax
mov    %eax,0x16c(%rip)        # 0x100001018 <natural_generator.b>

Here we add 1 to %eax and store the result back into memory. Let’s run these instructions and verify the result:

(gdb) nexti 2
(gdb) x/d &b
0x100001018 <natural_generator.b>:  0
(gdb) p (int)$rax
$15 = 0

The next two instructions set us up to return a + b:

mov    -0x4(%rbp),%eax
add    0x163(%rip),%eax        # 0x100001018 <natural_generator.b>

Here we load a into %eax and then add b. At this point, we’d expect %eax to be 1. Let’s verify:

(gdb) nexti 2
(gdb) p $rax
$16 = 1

%eax is used to store the return value from natural_generator, so we’re all set up for the epilogue which cleans up the stack and returns:

pop    %rbp
retq   

Now we understand how b is initialized, let’s see what happens when we run natural_generator again:

(gdb) continue
Continuing.
1

Breakpoint 1, natural_generator () at static.c:5
5           int a = 1;
1: x/i $pc  0x100000e94 <natural_generator+4>:  movl   $0x1,-0x4(%rbp)
(gdb) x &b
0x100001018 <natural_generator.b>:  0

Because b is not stored on the stack with other local variables, it’s still zero when natural_generator is called again. No matter how many times our generator is called, b will always retain its previous value. This is because it’s stored outside the stack and initialized when the loader moves the program into memory, rather than by any of our machine code.

Conclusion

We began by going over how to read assembly and how to disassemble a program with GDB. Afterwards, we covered how static local variables work, which we could not have done without disassembling our executable.

We spent a lot of time alternating between reading the assembly instructions and verifying our hypotheses in GDB. It may seem repetitive, but there’s a very important reason for doing things this way: the best way to learn something abstract is to make it more concrete, and one of the best way to make something more concrete is to use tools that let you peel back layers of abstraction. The best way to to learn these tools is to force yourself to use them until they’re second nature.

  1. You’ll notice we’re using Make to build `simple.c` without a makefile. We can do this because Make has implicit rules for building executables from C files. You can find more information about these rules in the [Make manual](http://www.gnu.org/software/make/manual/make.html#Implicit-Rules).

  2. You can also have GDB output Intel syntax, which is used by NASM, MASM, and other assemblers, but that’s outside the scope of this post.

  3. Processors with SIMD instruction sets like MMX and SSE for x86 and AltiVec for PowerPC will often contain some registers that are wider than the CPU architecture.

  4. A discussion of object formats, loaders, and linkers is best saved for a future blog post.

David albert 150
Tweet

Allison and Zach are Hacker Schoolers

Allison

Allison

Zach

Zach

We’re thrilled to announce that Allison Kaptur and Zach Allaun have agreed to join us as Hacker School facilitators!

Zach and Allison were both students in our summer batch, where they blew us away with their love of programming and how quickly and voraciously they learned. They’re exceptionally friendly people, with phenomenal social and communication skills. They also have a penchant for always going deeper, whether they’re hacking on the ClojureScript internals or studying the subtleties of C pointers. Plus, they’re just really fun people to work with.

Allison studied astrophysics at Yale, and then went to Wall St., where she built bank models and tried to wrangle Excel into doing large-scale time series analysis. Thankfully for us, she later fell in love with programming, and has since ditched finance to spend her days coding Python and C.

Zach went to college expecting to study political science, but quickly found computer science much more to his liking. After exhausting most of the CS classes his university offered, he decided to leave to study programming on his own. He’s since taken off like a rocket, and has already coauthored an essential piece of Hacker School infrastructure (a TrueSkill implementation in Clojure, which we use for our ping-pong tournaments.

We are unbelievably lucky to have found Allison and Zach, and we couldn’t be happier to have them as part of the team.

– Hacker School (Nick, Dave, Sonali, Alan, Tom, Allison, and Zach)

Learning C with gdb

Coming from a background in higher-level languages like Ruby, Scheme, or Haskell, learning C can be challenging. In addition to having to wrestle with C’s lower-level features like manual memory management and pointers, you have to make do without a REPL. Once you get used to exploratory programming in a REPL, having to deal with the write-compile-run loop is a bit of a bummer.

It occurred to me recently that I could use gdb as a pseudo-REPL for C. I’ve been experimenting with using gdb as a tool for learning C, rather than merely debugging C, and it’s a lot of fun.

My goal in this post is to show you that gdb is a great tool for learning C. I’ll introduce you to a few of my favorite gdb commands, and then I’ll demonstrate how you can use gdb to understand a notoriously tricky part of C: the difference between arrays and pointers.

An introduction to gdb

Start by creating the following little C program, minimal.c:

int main()
{
  int i = 1337;
  return 0;
}

Note that the program does nothing and has not a single printf statement.1 Behold the brave new world of learning C with gdb!

Compile it with the -g flag so that gdb has debug information to work with, and then feed it to gdb:

$ gcc -g minimal.c -o minimal
$ gdb minimal

You should now find yourself at a rather stark gdb prompt. I promised you a REPL, so here goes:

(gdb) print 1 + 2
$1 = 3

Amazing! print is a built-in gdb command that prints the evaluation of a C expression. If you’re unsure of what a gdb command does, try running help name-of-the-command at the gdb prompt.

Here’s a somewhat more interesting example:

(gbd) print (int) 2147483648
$2 = -2147483648

I’m going to ignore why 2147483648 == -2147483648; the point is that even arithmetic can be tricky in C, and gdb understands C arithmetic.

Let’s now set a breakpoint in the main function and start the program:

(gdb) break main
(gdb) run

The program is now paused on line 3, just before i gets initialized.Interestingly, even though i hasn’t been initialized yet, we can still lookat its value using the print commnd:

(gdb) print i
$3 = 32767

In C, the value of an uninitialized local variable is undefined, so gdb might print something different for you!

We can execute the current line with the next command:

(gdb) next
(gdb) print i
$4 = 1337

Examining memory with x

Variables in C label contiguous chunks of memory. A variable’s chunk is characterized by two numbers:

  1. The numerical address of the first byte in the chunk.

  2. The size of the chunk, measured in bytes. The size of a variable’s chunk is determined by the variable’s type.

One of the distinctive features of C is that you have direct access to a variable’s chunk of memory. The & operator computes a variable’s address, and the sizeof operator computes a variable’s size in memory.

You can play around with both concepts in gdb:

(gdb) print &i
$5 = (int *) 0x7fff5fbff584
(gdb) print sizeof(i)
$6 = 4

In words, this says that i’s chunk of memory starts at address 0x7fff5fbff5b4 and takes up four bytes of memory.

I mentioned above that a variable’s size in memory is determined by its type, and indeed, the sizeof operator can operate directly on types:

(gdb) print sizeof(int)
$7 = 4
(gdb) print sizeof(double)
$8 = 8

This means that, on my machine at least, int variables take up fourbytes of space and double variables take up eight.

Gdb comes with a powerful tool for directly examing memory: the x command. The x command examines memory, starting at a particular address. It comes with a number of formatting commands that provide precise control over how many bytes you’d like to examine and how you’d like to print them; when in doubt, try running help x at the gdb prompt.

The & operator computes a variable’s address, so that means we can feed &i to x and thereby take a look at the raw bytes underlying i’s value:

(gdb) x/4xb &i
0x7fff5fbff584: 0x39    0x05    0x00    0x00

The flags indicate that I want to examine 4 values, formatted as hex numerals, one byte at a time. I’ve chosen to examine four bytes because i’s size in memory is four bytes; the printout shows i’s raw byte-by-byte representation in memory.

One subtlety to bear in mind with raw byte-by-byte examinations is that on Intel machines, bytes are stored in “little-endian” order: unlike human notation, the least significant bytes of a number come first in memory.

One way to clarify the issue would be to give i a more interesting value and then re-examine its chunk of memory:

(gdb) set var i = 0x12345678
(gdb) x/4xb &i
0x7fff5fbff584: 0x78 0x56 0x34 0x12

Examining types with ptype

The ptype command might be my favorite command. It tells you the type of a C expression:

(gdb) ptype i
type = int
(gdb) ptype &i
type = int *
(gdb) ptype main
type = int (void)

Types in C can get complex but ptype allows you to explore them interactively.

Pointers and arrays

Arrays are a surprisingly subtle concept in C. The plan for this section is to write a simple program and then poke it in gdb until arrays start to make sense.

Code up the following arrays.c program:

int main()
{
    int a[] = {1,2,3};
    return 0;
}

Compile it with the -g flag, run it in gdb, then next over the initialization line:

$ gcc -g arrays.c -o arrays
$ gdb arrays
(gdb) break main
(gdb) run
(gdb) next

At this point you should be able to print the contents of a and examine its type:

(gdb) print a
$1 = {1, 2, 3}
(gdb) ptype a
type = int [3]

Now that our program is set up correctly in gdb, the first thing we should do is use x to see what a looks like under the hood:

(gdb) x/12xb &a
0x7fff5fbff56c: 0x01  0x00  0x00  0x00  0x02  0x00  0x00  0x00
0x7fff5fbff574: 0x03  0x00  0x00  0x00

This means that a’s chunk of memory starts at address 0x7fff5fbff5dc. The first four bytes store a[0], the next four store a[1], and the final four store a[2]. Indeed, you can check that sizeof knows that a’s size in memory is twelve bytes:

(gdb) print sizeof(a)
$2 = 12

At this point, arrays seem to be quite array-like. They have their own array-like types and store their members in a contiguous chunk of memory. However, in certain situations, arrays act a lot like pointers! For instance, we can do pointer arithmetic on a:

= preserve do
  :escaped
    (gdb) print a + 1
    $3 = (int *) 0x7fff5fbff570

In words, this says that a + 1 is a pointer to an int and holds the address 0x7fff5fbff570. At this point you should be reflexively passing pointers to the x command, so let’s see what happens:

= preserve do
  :escaped
    (gdb) x/4xb a + 1
    0x7fff5fbff570: 0x02  0x00  0x00  0x00

Note that 0x7fff5fbff570 is four more than 0x7fff5fbff56c, the address of a’s first byte in memory. Given that int values take up four bytes, this means that a + 1 points to a[1].

In fact, array indexing in C is syntactic sugar for pointer arithmetic: a[i] is equivalent to *(a + i). You can try this in gdb:

= preserve do
  :escaped
    (gdb) print a[0]
    $4 = 1
    (gdb) print *(a + 0)
    $5 = 1
    (gdb) print a[1]
    $6 = 2
    (gdb) print *(a + 1)
    $7 = 2
    (gdb) print a[2]
    $8 = 3
    (gdb) print *(a + 2)
    $9 = 3

We’ve seen that in some situations a acts like an array and in others it acts like a pointer to its first element. What’s going on?

The answer is that when an array name is used in a C expression, it “decays” to a pointer to the array’s first element. There are only two exceptions to this rule: when the array name is passed to sizeof and when the array name is passed to the & operator.2

The fact that a doesn’t decay to a pointer when passed to the & operator brings up an interesting question: is there a difference between the pointer that a decays to and &a?

Numerically, they both represent the same address:

= preserve do
  :escaped
    (gdb) x/4xb a
    0x7fff5fbff56c: 0x01  0x00  0x00  0x00
    (gdb) x/4xb &a
    0x7fff5fbff56c: 0x01  0x00  0x00  0x00

However, their types are different. We’ve already seen that the decayed value of a is a pointer to a’s first element; this must have type int *. As for the type of &a, we can ask gdb directly:

= preserve do
  :escaped
    (gdb) ptype &a
    type = int (*)[3]

In words, &a is a pointer to an array of three integers. This makes sense: a doesn’t decay when passed to &, and a has type int [3].

You can observe the distinction between a’s decayed value and &a by checking how they behave with respect to pointer arithmetic:

= preserve do
  :escaped
    (gdb) print a + 1
    $10 = (int *) 0x7fff5fbff570
    (gdb) print &a + 1
    $11 = (int (*)[3]) 0x7fff5fbff578

Note that adding 1 to a adds four to a’s address, whereas adding 1 to &a adds twelve!

The pointer that a actually decays to is &a[0]:

= preserve do
  :escaped
    (gdb) print &a[0]
    $11 = (int *) 0x7fff5fbff56c

Conclusion

Hopefully I’ve convinced you that gdb a neat exploratory environment for learning C. You can print the evaluation of expressions, examine raw bytes in memory, and tinker with the type system using ptype.

If you’d like to experiment further with using gdb to learn C, I have a few suggestions:

  1. Use gdb to work through the Ksplice pointer challenge.

  2. Investigate how structs are stored in memory. How do they compare to arrays?

  3. Use gdb’s disassemble command to learn assembly programming! A particularly fun exercise is to investigate how the function call stack works.

  4. Check out gdb’s “tui” mode, which provides a grahical ncurses layer on top of regular gdb. On OS X, you’ll likely need to install gdb from source.

Alan is a facilitator at Hacker School. He’d like to thank David Albert, Tom Ballinger, Nicholas Bergson-Shilcock, and Amy Dyer for helpful feedback.

Curious about Hacker School? Read about us, see what our alumni say, and apply to our fall batch.

  1. Depending on how aggressive your C compiler is about optimizing useless code, you may have to make it do something :) I tried running these examples on my Raspberry Pi and everything got compiled away.
  2. Formally, an array name is a “non-modifiable lvalue”. When used in an expression where an rvalue is required, an array name decays to a pointer to its first element. As for the exceptions, the & operator requires an lvalue and sizeof is just weird.
Alan odonnell 150
Tweet

Fall Applications and Hacker School Residencies

tl;dr: Applications for our fall batch are open. Etsy is sponsoring the batch and providing $5k grants for female students (we’re open to more sponsors). We’re starting a Hacker School Residency program, where awesome programmers come work closely with our students for one or two week stints. Peter Seibel will be the first resident. You can nominate residents.

What if?

Hacker School exists largely because we like to ask, “What if?” What if we ran a writer’s retreat for programmers? What if there were no grades, teachers, or formal curriculum? What if you could always feel comfortable saying, “I don’t understand?” What if we made social rules explicit?

What if Hacker School were half women?

Last batch, we partnered with Etsy to get equal numbers of men and women in Hacker School. We had no idea if that was feasible, since in our three previous batches we had had exactly one woman.

We were blown away by the response we got: Over 650 women applied, and we ended up with many more qualified women requesting financial assistance than we anticipated. Thankfully, 37signals and Yammer stepped up to provide additional grants for women.

The result? Twenty three of the 51 students in our current batch are female, and Hacker School is much better for it.

One thing we’ve heard repeatedly from women in Hacker School is that they love that here they’re “programmers” and not “female programmers.” A student told me last week that it feels like Hacker School has always been half women. It’s no longer a “what if.”

We’re thrilled to announce today that Etsy will be hosting our fall batch and sponsoring an additional 10 grants. Etsy has been an absolute pleasure to work with, and it’s hard to imagine how they could have been any more thoughtful, tasteful, or respectful. They’ve been an ideal partner.

We want to make this batch bigger and better than our last, and we’re currently in talks with several other companies interested in sponsoring Hacker School and providing grants. If you’re interested in working with us as well, please let us know.

Another “what if”

Some of the best moments at Hacker School have come after talks by guest speakers.1 David Nolen spoke one Saturday about ClojureScript. The Q&A after his talk turned into a mini workshop with a handful of students, which turned into David staying and sharing his expertise and enthusiasm for nearly eight hours. He came back a few weeks later and walked a group of students through the internals of the ClojureScript compiler until almost 11pm. That led to half a dozen Hacker Schoolers signing Clojure contributor agreements and adding support for code reflection to the ClojureScript REPL.

My cofounder Dave recently asked another “what if”: What if the world’s best programmers did two week residencies at Hacker School? As usual, we don’t know the answer (which is why it’s a question worth asking), but we suspect it will be good.

Admittedly, this sounds far-fetched. Why should the best programmers in the world take two weeks off from their jobs to come work with us? But Hacker School itself initially sounded improbable. Who would quit their jobs and dedicate themselves to becoming better programmers for three months, and where would we find space to do it – full-time, for free – in one of the most expensive real estate markets in the world?

It no longer seems improbable that people would leave their jobs to do Hacker School. In fact, dozens of people now have, and many have done much more. Well over half of our current batch moved to New York solely for Hacker School, and more than 20% of our current batch moved countries to do it. (And somehow we’ve managed to beg, borrow, or steal space for five batches.) While Hacker School may have sounded improbable when we started, it clearly wasn’t impossible.

So today we’re announcing our Hacker School Residency. The idea is simple: Great programmers come work with us, full-time, on open source projects.

We’re delighted to say that Peter Seibel, author of Coders at Work and Practical Common Lisp (and programmer at Etsy), has generously agreed to be our first resident. Peter spoke at Hacker School earlier this summer and blew us away with his thoughtful advice for becoming a better programmer and genuine engagement with students.

We’re looking for more Hacker School Residents now. We’re looking for people who are at the top of their field, love what they do, and are genuinely nice human beings. If you’d like to nominate someone, including yourself (especially if your name is Peter Norvig), please send us a nomination. If you think it’d be impossible to take an extra week or two off work, ask yourself: “What if?” Maybe we can find a way to make it happen.

Always an experiment

A student asked me recently how I thought Hacker School was going. It was a surprisingly hard question to answer. On the one hand, I’m unabashedly proud of what we’ve accomplished in just over a year. On the other hand, we’re chronically dissatisfied with Hacker School: There are a thousand ways we think we could do better, and we’re far from where we want to be.

To be clear, we’re not a bootcamp or a training program, we’re something different: The place where programming is important, where people come to focus, learn from each other, and do great work. We’re an ongoing experiment, and we think we can keep getting better, so long as we’re always willing to ask, “what if?”

Curious? Read about Hacker School, see what our alumni say, and apply to our fall batch.

  1. This is a separate post of its own, but the short of it is: Hacker School talks are 20 minutes, with no questions during the talk. After the talk we take a two minute break, at which point it’s entirely socially acceptable for anyone who wants to to leave. Then we start Q&A and go until students run out of questions or the speaker can’t take any more, usually the latter.

Welcome Tom and Alan!

We’ve got exciting news: We’ve hired Tom Ballinger and Alan O'Donnell to be facilitators for the next batch of Hacker School! I can’t describe how happy we are to have them join us.

Alan was in the first batch and has been part of Hacker School since the very beginning. He’s a polyglot and enjoys coding in everything from C to Clojure. Unsatisfied with the mathematical purity of Haskell, he’s recently started learning Coq, a language so pure it’s not even Turing complete.

Tom was in the most recent batch and was a tremendous asset to the group. I checked my notes from when I interviewed him for the batch, and I had exactly one concern: Tom was so excited about doing Hacker School I was worried we wouldn’t live up to his expectations. Given that he’s agreed to join us, I guess we did ok.

Tom and Alan embody the best of Hacker School: They love programming and are smart, intensely intellectually curious, and extremely friendly. They’re also proof that you don’t need to study computer science to become a great programmer. In fact, neither of them were CS majors or followed a traditional path to programming.

I can’t think of a single time I’ve heard either of them say something mean. On the other hand, I can think of countless times I’ve heard them ask thoughtful questions, or go out of their way to help others improve their understanding. They’re egoless programmers; their focus is always on helping themselves and others learn, and never on showing off how much they already know.

For these and many more reasons, we know that they’ll be fantastic facilitators. They have Hacker School in their DNA.

Please join Dave, Sonali and me in welcoming Tom and Alan. Together we’re going to continue to build Hacker School into the best community to learn and grow as a programmer.

Now back to reviewing applications for the next batch!

– Hacker School Nick, Dave, Sonali, Tom and Alan

P.S. We think it’s important to have both male and female facilitators, and we’re cognizant of the fact that we just hired two more dudes. We also feel strongly about only hiring people who truly grok Hacker School, and to do that they need to have experienced it first hand. We’ll be hiring more women as the gender balance of our alumni improves.

Do you have enough experience for Hacker School?

People often ask how much programming experience you need to do Hacker School. We think that’s the wrong question to ask, because we don’t actually care how much experience you have. What we really care about is if you’d be a good fit for Hacker School and if Hacker School would be a good fit for you. We determine this primarily by figuring out if you love programming, so if you’re wondering if you should apply, the right question to ask yourself is, “do I really love programming?” If you do, you’d love Hacker School. If you don’t, you’d be unhappy here.

We can’t stress this enough: We spend almost all of our time at Hacker School coding. If you get excited thinking about how programming languages are written, spend hours on end solving Project Euler problems for fun, and get a burst of joy every time you squash a bug, you’re almost certainly a good fit for Hacker School. If you don’t enjoy these things, you won’t be happy here and you won’t get what you want out of it.

It may seem like we’ve sidestepped the original question, so I’ll answer it in a different way: How much programming experience do you need to do Hacker School? Enough to know you love it. This doesn’t have to be very much, although it can’t be nothing, because you can’t know you love something if you’ve never done it before. We’ve met people who have fallen in love with programing after as little as a few months. So don’t concentrate on how long you’ve been programming. We won’t be. We’ll be concentrating on how much you love it, and what you’ve done in the time you’ve been coding.

Try not to see this as “applying” to Hacker School. There aren’t any bars you have to get over, nor are there hard qualifications that we look for. Instead, see it as figuring out if Hacker School is what you really want. Read our about page and our FAQ. We try to be as explicit as possible to make it easy for you to figure out if you’ll enjoy and benefit from Hacker School.

Don’t fool yourself into thinking that you love programming if you don’t really. You’ll only end up unhappy down the road. Don’t lie to us and say that you really love programming if you don’t. Hacker School won’t make it easier for you to start your own startup or get a high paying job as a programmer if you don’t actually enjoy programming. You simply won’t get what you want out of it.

If you haven’t programmed before, there’s no time like now to start! There are lots of good resources online. We recommend getting your feet wet with Codecademy’s JavaScript lessons and moving on to Learn Python the Hard Way or Learn Ruby the Hard Way by Zed Shaw. Once you know a bit, you can have some fun solving Project Euler problems. They start out easy but get challenging quickly.

Have fun and do what you love. If that happens to be programming, you should come to Hacker School.

David albert 150
Tweet

Summer 2012 Applications Open

This will be a long post, so we’ll cut to the chase for those who just want the vital information:

Applications are open for the Summer 2012 batch of Hacker School. The deadline to apply is May 7, though we’ll accept late applications. It will run from June 4 to August 25. The batch will be hosted at Etsy, which is also sponsoring $50,000 worth of scholarships for female programmers (Hacker School remains free for all students; the scholarship money is for living expenses). We’re trying to make this batch at least 50% women.

The summer batch

We’re hoping to make our summer batch our biggest yet with 40 students. That number would have seemed absurd to us last year when we started with six students, but now it seems quite doable.

We’re going to run the batch in two cohorts of 20 students each. We think this will give people the best of all worlds. Day to day it will feel a lot like this batch, which has 20 students, but we’ll have social and other events where the two groups come together, and they’ll be able to draw on the collective intelligence of a much larger group.

Etsy

Marc Hedlund, Etsy’s VP of Engineering, reached out earlier this year to say he had an idea for a partnership between Hacker School and Etsy. Marc’s idea was awesome: Etsy would host and sponsor the next batch of Hacker School and provide scholarships to get as many qualified women into the batch as possible.

So here’s the deal: In addition to space, Etsy is offering ten $5,000 scholarships for women to do Hacker School. The scholarships will be given based on financial need. “Need” will be determined based on an honor code—if you say doing Hacker School would be a real financial hardship, you’ll get the money. You can spend the money on whatever expenses necessary to free you up for Hacker School, no questions asked.

We hope this will help in achieving one of our and Etsy’s primary goals for this batch: To get the percentage of female students to be at least 50%.

Women programmers

I’ve spent the last several years in heavily male-dominated environments. Dave and I studied computer science and electrical engineering, which were over 90% male. I graduated and joined a 100%-male engineering team. Dave and I were in the Summer 2010 batch of Y Combinator with 80-some other founders but only a handful of women.

On the flip side, I’ve spent time in heavily female-dominated settings. In college, I was one of two men on the 17-person executive board of the Women’s Center, and helped organize Take Back the Night marches for five years. Those were eye-opening experiences—as a straight, white male, I don’t spend a lot of time being in the minority.

I bring these experiences up because I think about them whenever I’m in a room of programmers and there’s only one woman. No matter how welcoming and friendly the environment, you burn at least a few cycles being cognizant of the fact you’re “different” from most of the people around you. At least I know that’s the way my brain works.

We’re not going to lower the bar for female applicants. It frustrates us a little that we feel the need to say that, and we think it underlines the sexism (intentional and not) that so pervades the programming world.

But we want to say that now, so people don’t have to waste time asking or debating the point. Women will be judged on the exact same scale as men. We think to do otherwise would be insulting and counterproductive. We care a lot about getting more women into Hacker School, but we won’t do it at the expense of the quality of the batch.

A home for lonely programmers

One thing we’ve heard again and again from students is that they were lonely before Hacker School, because few or none of their friends before Hacker School were programmers. When they learned a new bash trick, or experimented with a new programming language, they had no one who’d care to share it with. We’re proud that Hacker School has become an intellectual home for such programmers.

But if it’s sometimes lonely to be a male programmer, it must be doubly so for female hackers. We believe there are way more than 20 women out there who love coding like we do, and who dream of spending three months with a group of smart, intellectually curious people who love programming. Especially if they know they won’t be the only woman in the room.

We believe there are female programmers out there who love and want to learn more about C and Unix and Haskell and coroutines, and who are tired of showing up to “women in tech” events only to find all the other women there are in design and marketing.

Our environment

We strive to make Hacker School the best place in the world to level up as a programmer.

There are very few rules at Hacker School. That’s by design, because we don’t like rules ourselves, and we don’t believe coercion is conducive to learning.

The few rules we do have are social and aim to fix some of the bugs we’ve found in other communities. Social conventions govern most of the world, but they’re usually unspoken. We make some social rules explicit at Hacker School because we’ve found it makes for a better environment. Two of these rules are “No well-actuallys” and “no feigning surprise.”

No “well, actuallys” means you shouldn’t correct minor inaccuracies in things others say when it has nothing to do with the conversation at hand. If you code something that’s almost but not entirely correct, it won’t compile. 1 If you say something in conversation that’s almost but not entirely correct, it probably doesn’t matter. Programmers have a bad habit of saying, “well, actually…” when people do the latter. That can be obnoxious, particularly when the inaccuracy is irrelevant to what’s being discussed.

No feigning surprise means you shouldn’t act surprised when you find out someone doesn’t know something, even if you think it’s “obvious.” If someone asks, “Who’s RMS?” don’t say, “You don’t know who Richard Stallman is?!” It adds no value, and only serves to make one person feel better by making the other person feel worse.

We don’t have these rules to make Hacker School “female-friendly.” We have these rules because we think they make Hacker School human-friendly. We have them because they help remove the ego and fear of embarrassment that so frequently get in the way of education. They help make Hacker School a place where we ourselves want to learn.

An experiment

Like every batch, this one is an experiment, both for us and for Etsy. We don’t know if we’ll be able to get 20 qualified female students, but we do know that the only way to find out is to try.

If three months of intense coding sounds euphoric to you, you should apply to Hacker School. You can read our about page, to learn more.

  1. If you read that and thought, “that’s not true, you could also get a runtime or logical error,” you’ve responded like a typical programmer. It’s also a perfect example of a well-actually. Hat tip to Miguel de Icaza for coining the term.