Faith in the Feds? I don’t have much, so I expected a close enough for government work City, cleansed by apathy and sloth of all but the coarsest detail. However, surprise surprise…

Out popped a precise rendition of San Francisco’s crisp-yet-quirky street grid, the sinuous curves of 101 and 280, and every dock and marina on the waterfront, faithfully modeled down to the individual slips. Hubba hubba!

In tribute to the Census Bureau’s hard work – and as a study of how we occupy, manipulate, and subdivide our surroundings – I humbly present these maps, wherein each Census block is colored to contrast with its neighbors and contribute harmoniously to the overall texture. Below, you’ll find New York City, Boston, Seattle, and thirteen other interesting places. Click to embiggen, as always:

At this point, I’d be remiss not to mention artist Jennifer Maravillas and her kindred drawings of Manhattan and San Francisco… created entirely by hand. Wowzers.

Bravo, Jen, and block on, y’all!

West out of quaint Honokaʻa, the road rolls gracefully through pasture and rainforest. Sure, it’s a bit hilly. But extreme? No way. Did I miss a turn somewhere? Then, ten miles in, after a particularly worrisome run of flatness, the faint hum of tourists, a parking lot, and the pavement dives into a giant gash. I’d reached the rim of the Waipiʻo Valley, and it was all downhill from here:

Per the signage, by law, only bona fide four wheel drive vehicles can proceed. No AWD allowed! Always the good citizen, I curbed my wimpy rental SUV, unpacked my level, and walked over the lip.

By any measure, the Waipiʻo Valley Road is steep: you might say damn steep, extremely steep, or even butt-ass f*cking steep, depending upon your particular level of refinement and color commentary. It averages a 26% grade – descending 900 vertical feet over two thirds of a horizontal mile – and long sections tilt at a sustained 30% plus. In general, for whatever reason, photos don’t do steep streets justice, but here’s one anyways:

As I descended, gravity urged: “Run, roll, slide to the bottom!” But that would hurt. Instead, I carefully checked my acceleration with every step. Five minutes in, perspiration began to bead on my forehead. As I walked down the hill, the constant effort required to arrest my fall was making me sweat!

Along the way, I measured the instantaneous pitch with my digital level: 36%, 33%, 37%, 39%… and nothing steeper, anywhere along the way, top to bottom.

Now, back to San Francisco, the median grade of the northwest quarter of Bradford Street’s tiltiest slab, as surveyed by Yours Truly, last November:

Yes, that’s a slope of 42.8%. Take a deep breath, Bradford, because in the kingdom of wheelbase tilt – how much your car would pitch, axle-to-axle, while driving a valid path up or down the road – you’re still the Big Kahuna.

Now, please permit me to veer off topic and dish a few niblets of Kona (leeward Big Island) wisdom:

• Shaka: Like the Californian hang-loose sign, but with back of hand towards the recipient. Meaning: hey brah, right on, thanks, etc. If you’re a tourist and get a shaka from a local, you’re doing it right.
• Luau: Tried the Royal Kona version. Leis, fire, swords, poi, pork, and mai tais. Serviceable portrayal of Pacific Island dances. Pleasant with a hint of shlock. Recommended.
• Accommodations: Relative to Waikiki, main town Kailua is cute and undespoiled, but I refuse to bed down anywhere within walking distance of a Bubba Gump’s Shrimp Company. To get off the tourist grid, travel 15 miles south to Captain Cook and the low-frills Manago Hotel. No TV in the rooms!
• Driving: Big Island is Big. But surely, I canz haul ass on the “Belt Highway”? Bzzzt! Even the slightest curve in the road means 45mph, tops. Any attempt to not slow down and feel the aloha will be met by a stinky Datsun pickup, limping just off your front bumper.
• Snorkeling: We snorkeled Kona’s most popular sites: Kealakekua Bay (incredible variety of sea life), Honaunau’s Two Step (spinner dolphins), and Kahaluʻu (fish-bowl-style shallows). The young lava shoreline means many holes, fingers, slots, arches, cracks, tubes. Go before 9am to avoid crowds and near-entry gridlock. Equipment: dry snorkel with a purge valve, water shoes, fins. Visit a dive shop to find a mask that fits right. Cover your back with sunscreen or shirt. After you’ve “gotten your feet wet”, try the more adventurous options.
• Helicopter: “You don’t appreciate the full scale of the eruption until you see it from the air,” says my wife. And the electric lava sloshing in the two-hundred-foot-wide bathtub atop Puʻu ʻŌʻō crater… beyond words. We flew Safari Helicopters out of Hilo, but next time, I might try Blue Hawaiian.
• Luggage: Departing the Bay Area, our two bags weighed 92 pounds total. On the return, the same stuff tipped the scales at 108 pounds. The difference: damp, sandy clothes and gear? Dunno, but to avoid the onerous overweight fees, pack an extra bag, just in case.
• Island Fever: Primary symptom: distant, removed stare, manifesting hourly upon return to the mainland. “I think Steve’s in Hawaii again.”

Aloha!

When it comes to weather, past performance is often indicative of future results, so I charted the last fifty years of rainfall in downtown San Francisco:

That’s a line per rain year – the period running from July 1st through the following June, centered on California’s wet winter months – with a circle for each day sized by the amount of precipitation. The bars show the total rainfall received on either side of Valentine’s Day for each year. To enhance readability, smaller amounts are tinted “dead grass” brown and larger amounts “pooling water” blue. The data comes from meteorologist Jan Null’s Golden Gate Weather site, with updates from the NWS monthly summaries.

On average, downtown San Francisco gets about 22.0 inches of annual rainfall: roughly two thirds (14.8″) on or before February 14th and the remainder (7.2″) after. This season thus far, the City has received 6.8 inches, about 46% of normal for the date, ranking as the third driest pre-Valentine’s period on record since 1960, behind 1975-76 and 1990-91.

As for our March Miracle? Check the chart, and you’ll find that the years which finished wettest share one thing in common: none started out unusually dry. Furthermore, the ten seasons with the driest beginnings averaged 8.1 inches before V-Day and a paltry 5.2 inches after, which suggests that when the last raindrop falls, the 2011-12 rain year will clock in at around half of normal.

“Don’t call a drought!” implore my skis and lawn. And perhaps rightfully so. At any moment, Mother Nature could deliver a duck-drowning deluge of Noachian proportions. However, the numbers suggest the opposite scenario, wherein the coastal Californian might be wise to toodle over to the local Home Despot and get one of them new fangled home desalination units – while the gettin’s good!

Hello there, friend. I’m the country. Chances are, within the past week or three, you flew over or motored right through me. And that’s okay, of course – life’s hectic, and it’s a long way from here to there. But I miss you.

So, please, come and visit. Upon arrival, you can do whatever you want. Relax, discover yourself, commune with nature, spot a UFO, fire large caliber weapons… I won’t judge.

Getting here is an easy two-step process. First, figure out where everyone else is. Then, head in the other direction:

That’s me in black: every U.S. 2010 Census block with less than one resident per square mile. I freckle the East, define the West, and blanket icy Alaska almost in its entirety:

Perhaps grizzlies and tundra aren’t your thing? Bask in the warmth of my untrammeled Hawaiian shores:

Of the eight main islands, Lānaʻi is for lovers of peace and solitude. Unless you prefer to tiptoe through the UXO.

All in all, stateside, just under 155,000 people live in my 2,120,000 square miles: 0.05% of the total U.S. population spread across 60% of the land. In other words, when you visit, you’ll have plenty of room to stretch, pitch a tent, and build a campfire. Or bang on the bongos, strip naked, and prance around like an antelope. Whatever makes you happy.

Looking forward to seeing you soon!

The Country

In these charts, ranked choice ballot trajectories radiate from the center, first round inside to last round out. Angular width represents the fraction of the overall vote, like a pie graph. An additional 1,900 votes have straggled in since we created the originals, causing Hall and Alioto-Pier’s exits to flip-flop and the color scheme to change slightly.

First, let’s view all ballots with a valid #1 choice other than mayor-elect Ed Lee or runner-up John Avalos, grouped by initial preference:

Next, the same non-Lee/Avalos first choice ballots, grouped by final result:

Finally, the 27% of all ballots cast, free of Lee or Avalos in any spot, which became exhausted:

Tom Petty was right! In a 16-candidate election, the wasting is the hardest part.

And that’s all I have to say about that.

My my, San Francisco Department of Elections, what a big table you have. Because the whole circus – sixteen (16!) people – decided to run for mayor this year, and the City uses the system known as ranked choice voting.

For those unfamiliar, here’s how ranked choice voting works. Go to a polling place, grab a ballot, and mark your favorite candidates in order of preference, one, two, three. First choice is Abraham Lincoln, of course. Second choice, Betsy Ross. Third choice, Winston Churchill. San Francisco elections are pick three, so you’re done. Bask in the warm glow of civic duty.

But don’t bask long, because it’s time to tabulate! Add up the first choice votes. A majority wins, but otherwise, determine the least popular candidate (*), eliminate them, and give their votes to the next remaining choice on the supporting ballots. Repeat the process until finished.

In our hypothetical four-way election, suppose the first choices break out 40% Ross, 36% Lincoln, 18% Churchill, and 6% Gandhi. Gandhi is lowest, so he’s eliminated, and his supporters’ second choices split between Lincoln and Churchill. So, going into Round 2, we have 40% Ross, 39% Lincoln, and 21% Churchill. Churchill loses, and per next choices, two thirds of his votes go to Lincoln, and the rest to Ross. Honest Abe wins with a 53% majority.

Now, ramp that up to 16-candidate Frisco, and you get the tabular thicket of numbers pictured above.

In the interest of clarity, Burrito Justice tweeted:

Thanks! Hey @datapointed, time to apply the crayola chart methodology! RT @McClure_SF: @burritojustice http://www.sfelections.org/results/20111108/data/mayor.html via @kevinmonty

Great idea! Then, I whipped up a few drafts which didn’t go anywhere, and, at that point, I shoulda stopped. But Señor Justice had me hooked…

So, using the official San Francisco mayoral election data (as of November 17th), we calculated the trajectory of each ballot: the sequence of candidates that it counted towards in each round of the tabulation process. Next, we eliminated the trajectories for ballots with no valid choice. Then, we rendered each as a fixed-thickness horizontal line, colored by candidate, first round left to last round right. Finally, we stacked them to create a chart where the vertical axis represents the overall fraction of the voting pool.

Here are the trajectories, sorted most common first:

At the top, we see the purple and medium blue of winner Ed Lee and runner-up John Avalos. A first choice of either of those two counted for them all the way through. Next is the bright cyan of Dennis Herrera, dark blue for David Chiu, cream for Leland Yee, and so on. To decode the colors, click through to the large version, which has labels for the major candidates.

The same trajectories, grouped by first choice:

The bars along the left side show the initial preference by candidate. And… Newsflash! San Francisco isn’t completely colorblind; note the flow of votes along ethnic channels, especially from Herrera to Avalos and Chiu to Lee.

And to wrap it up, let’s sort by final result:

To the right, we see Lee over Avalos, and large slug of dark gray, representing the trajectories which had no choices left in the race. These exhausted ballots – 27% of the total cast – ended up not counting. That being said, in a sixteen candidate torture test, ranked choice voting trounces the traditional plurality system, which would have degenerated into a government-sponsored festival of wasted votes.

Nevertheless, the “27% exhausted” figure bothers me, so I have to ask:

Is there a fairer way to deal with the insane variety of San Francisco politics? A ranked choice primary to narrow the field, followed by a ranked choice runoff?

Let’s take a vote.

Footnote: (*) Technically, in San Francisco, in each tabulation round, the most popular candidate who has more votes than the total for all less popular candidates is determined, and the votes of these less popular candidates are redistributed simultaneously. Typically, one candidate’s votes are redistributed in each round, except in Round 4 of this election, in which four candidates votes were redistributed.

Watch daylight ebb and flow, fog linger and break, and storms bluster and clear in Ken Murphy’s wonderful A History Of The Sky: a calendar-style time-lapse chronicle of the heavens above San Francisco. Learn more from his site, and be sure to immerse yourself in the HD version of his latest full-year video (above).

[A History Of The Sky]

Rarely is something so squarely up my alley as DARPA’s Shredder Challenge. Simply download five shredded documents, reassemble the puzzles within, submit your answers, and claim a share of the $50,000 prize.

And sure, the Challenge’s sponsor – a research arm of the military-industrial complex – can use the resulting insights to pry into your sensitive affairs. But, ultimately, improved document destruction technologies will trickle down to the people, and privacy wins. So, register as a participant, grab the pieces, and totally unshred, dude!

For the aspiring long-term winner – aka the advantage player – it goes something like this. First you memorize Basic Strategy: whether to hit, split, double down, or stand, based solely upon the value of your hand and the dealer’s upcard. Hit sixteen versus seven. Always split aces, never fives. Double ten against nine or less. And so on. In a single deck game, well-executed basic strategy brings you neck-and-neck with the house, advantage-wise.

Then, as seen in the movies, you learn to count cards. Statistically, a blackjack player wins the most money when the deck contains more high cards than low. And thus, the Hi-Lo system, which assigns -1 to tens, jacks, queens, kings, and aces and +1 to sixes and under. Start the count at zero after each shuffle. Add the Hi-Lo value for each card as it comes out. Bet larger amounts as the count goes positive, and the minimum otherwise. Practice ’til it’s automatic. And voilà, you’ve become a bona fide card counter, operating at one-percent advantage, ready to bring down the house at your earliest convenience.

Except for one gotcha: the casinos are wise to your action. The dealers and pit crew can count too and, by Nevada law, eject anyone for any reason. If they suspect you’ve got skills, expect to be tapped on the shoulder, escorted to the door, and launched through it.

So, as a countermeasure, you cultivate your cover. Should you look like some Silicon Valley automaton, surveiling the table and stacking chips to the count? No, probably not. A drunken rube? Flush with bravado and short on sense? Pissing away his inheritance? Yes yes yes!

Enter the shuffle, which, in an ideal world, takes an ordered deck (top) and completely scrambles it (bottom):

The colored blocks represent cards, and lines between decks show their trajectories. At the top and bottom, curves link cards that appeared back-to-back in the unshuffled deck – thin before to thick after.

Mathematicians say that it takes at least seven riffles – the two-stack interleave most people call a “shuffle” – to randomize a deck. However, while the dealer fiddles with the cards, the casino doesn’t make money, so to maximize the take, corners get cut.

From Cal-Reno blackjack trips of yore, I recall the very popular RRSR shuffle: two riffles, a strip (a series of running cuts), and riffle. Let’s simulate it with Epstein’s card interleave odds:

At first glance, it looks scrambled. However, in the curves along the bottom, we see a disproportionate number of short links deeper into the deck: pre-shuffle neighbors, still in the same relative order, now spread apart by seven cards, plus or minus a few. By golly, what we’ve got here is a failure to decorrelate.

Now, suppose we’re watching the discard pile, and the Jack of Diamonds goes in on top of the Queen of Hearts. After the next RRSR shuffle, when the Jack of Diamonds makes an appearance, there’s a good chance that the Queen of Hearts will soon follow. And therein lies the opportunity: to track clumps, predict aces, and engage in other profitable but presumably-pointless plays that soothe the looming pit boss as they fly under his radar.

So, hi ho, it’s off to scout casinos I go.

Note that my intended antics are 100% legal and completely Kosher, based solely upon what’s visible on the table. No monkey business like marking cards, bribing the dealer, toe-actuated in-shoe computers, etc. There will, however, be lots of free drinks and color commentary.

More details after I gross the first $1,000,000 – or get dragged off to the boiler room, cuffed to a pipe, and knocked sloppy. Whichever comes first!

That’s the contiguous United States, colored at each point by the zoom level of the smallest enclosing Google Maps tile wherein 1,000 or more people live, according to 2010 block-level Census data. Periwinkle represents zoom level 14+, navy blue 13, yellow 12, dark green 11, and so on.

The über-geek backstory? Last Friday, I told my Data Elves to get slippy – and create interactive Map X, featuring our own custom imagery, zoomable from continental down to “city core” scale, five-or-so miles across the browser window.

Eager to scope the project, the Elves grabbed an envelope, flipped it over, and made some calculations. We’d be rendering down to Google’s zoom level 14, and to cover the entire world, how many map tiles would we need to create? 357,000,000… each a 256×256 pixel image. The room buzzed with squeaky chatter about the “cloud” and FedEx-ing hard drives.

“Wait a sec,” I said, “Map X is U.S. only.” A mere 10,000,000 tiles would fill the bounding boxes of the Lower 48, Alaska, and Hawaii. Which certainly was a more manageable number, but still enough to make my compute farm sweat.

Could we do better? Yes!

Because, for the subject matter of Map X, detail correlates with population, and we could focus on the places where people live. Thus directed, the Elves mashed Census data with tile geography, applied the 1,000+ “interestingness” criteria, and the above image was born.

My team selected the old-school color scheme to maximize contrast whilst maintaining some semblance of readability. According to them, if it dredges up any repressed 8-bit feelings – like the urge to snipe an Atari 2600 off Ebay – that’s not a bug. It’s a feature.

Finally, the Elves added some padding – five tiles horizontally and four vertically – for the sake of context and to prevent solitary areas of interest from being marooned in “no imagery available at this zoom level”:

This scheme renders everything near urban areas in full detail, while displaying the most desolate pockets of the American West at enough magnification to keep 99.44% of the three people who look out there happy. Total number of tiles: a cool 1,000,000.

The takeaway? Those crafty Elves pruned the naïve Map X render by a factor of ten, earning them their quarterly bonus: seven nights on Maui, all expenses paid. Aloha, little buddies!