Links: 5/28
Business / Government / Management / News
Electric Scooter Charger Culture Is Out of Control
Every afternoon around 4 p.m., when school lets out, Brandon, an 18-year-old high-school senior in Los Angeles who asked to be referred to only by his first name, goes “Bird hunting.” He heads for his minivan and, on the drive home, he’ll swing through convenient neighborhoods, picking up about 13 Bird electric scooters along the way, tossing them into the back of his car.
“I have a whole system,” he says. “I’ll go home, put the 13 I initially caught on the chargers. They’ll charge for about three hours until around 7 or 8 p.m.” — when Bird makes more scooters available for charger pickup. “Then I’ll go back out.”
Over the course of the next few hours, Brandon loops around his Santa Monica, California, neighborhood collecting as many scooters as possible. He brings back his bounty and, as his parents sleep, neatly sets them up to charge in batches overnight.
The next morning he wakes up early, eats breakfast, and drops them off in groups of three at designated Bird Nests, designated pickup areas for scooters, on his way to school. For performing this service, Bird pays Brandon, a contract worker, up to several hundred dollars a night. On one particularly successful night, Brandon brought home $600.
- This is fascinating. And I had no clue that any of this existed.
- It’s amazing how technology is allowing us to play out so many economic theories in real scenarios. And quite quickly at that.
Strategy & Implementation of Third-Party Connections in Products
Building a product that connects to multiple third-party products is a common approach — an annotated twitter thread exploring strategic and execution challenges.
- I love articles about strategies like this.
- Specific to this topic I learned a ton here even though a good chunk I’ve hit before in building APIs myself. But Steven brings up some other issues to be aware of and do the mental walkthrough of side-effects, impact, gaming, etc.
Systems / Infrastructure / Cloud
Containers, Security, and Echo Chambers
There seems to be some confusion around sandboxing containers as of late, mostly because of the recent launch of gvisor. Before I get into the body of this post I would like to make one thing clear. I have no problem with gvisor itself. I think it is very technically “cool.” I do have a problem with the messaging around it and marketing.
There is a large amount of ignorance towards the existing defaults to make containers secure. Which is crazy since I have written many blog posts on it and given many talks on the subject. But I digress, let’s focus on the part of the README that mentions sandboxing with SELinux, Seccomp, and Apparmor. It says: “However, in practice it can be extremely difficult (if not impossible) to reliably define a policy for arbitrary, previously unknown applications, making this approach challenging to apply universally.”
Greetings. Reporting for duty. Literally I am the person who can do that. I was the person who did do that. I added the default Seccomp profile to Docker and maintained the default Apparmor profile. I have also done A LOT of research with regard to Linux kernel isolation and making containers secure. I also literally reported for duty, two years ago and made the patch to add the Seccomp annotation to Kubernetes… with the hopes of eventually turning on a default filter.
All big organizations have problems with “not invented here.” I tried my very best to inform everyone how these sandboxing mechanisms work but I am going to try one last time here.
- You may not believe this but our industry is full of politics, marketing FUD and all of the other normal crap you see everywhere else. As a matter of fact, I might say our industry has more of this than other industries. Why do I think this? Because we’re often attempting to disrupt, destroy or create new markets, movements, ecosystems, etc. Thus we are constantly hit by folks pushing their agendas. And we all have them. The key is to find the facts and incentives behind the agendas. If you’re hoping to avoid this type of stuff then technology may not be the place for you. Certainly not infrastructure, language design, blockchain/crypto, AI or social. :)
Full Cycle Developers at Netflix — Operate What You Build
The year was 2012 and operating a critical service at Netflix was laborious. Deployments felt like walking through wet sand. Canarying was devolving into verifying endurance (“nothing broke after one week of canarying, let’s push it”) rather than correct functionality. Researching issues felt like bouncing a rubber ball between teams, hard to catch the root cause and harder yet to stop from bouncing between one another. All of these were signs that changes were needed.
Fast forward to 2018. Netflix has grown to 125M global members enjoying 140M+ hours of viewing per day. We’ve invested significantly in improving the development and operations story for our engineering teams. Along the way we’ve experimented with many approaches to building and operating our services. We’d like to share one approach, including its pros and cons, that is relatively common within Netflix. We hope that sharing our experiences inspires others to debate the alternatives and learn from our journey.
- This aligns with much of our experience and mindset.
- We hope to write more on this soon.
Quick Hits:
- Tips for High Availability
- Exposing Container Services to a Bridge Network
- Google Kubernetes Engine 1.10 is generally available and ready for the enterprise
- Want to Debug Latency?
- SafeKeeper: protecting web passwords using trusted execution environments
- Workload Frequency Scaling Law — Derivation and Verification
Programming
Semantics and complexity of GraphQL
Semantics and complexity of GraphQL Hartig & Pérez, WWW’18
(If you don’t have ACM Digital Library access, the paper can be accessed either by following the link above directly from The Morning Paper blog site, or from the WWW 2018 proceedings page).
GraphQL has been gathering good momentum since Facebook open sourced it in 2015, so I was very interested to see this paper from Hartig and Pérez exploring its properties.
One of the main advantages (of GraphQL) is its ability to define precisely the data you want, replacing multiple REST requests with a single call…
One of the most interesting questions here is what if you make a public-facing GraphQL-based API (as e.g. GitHub have done), and then the data that people ask for happens to be very expensive to compute in space and time?
- This is really interesting and hits on a key area. How do you best keep a GraphQL query from bottlenecking your system by requesting a lot of data, perhaps unknowingly. The key points are towards the end:
What all this means practically is that the size of the response is everything. If we can find ways to limit the size of the response then we can bound time and space consumption.
- So how can we find the necessary limits:
Putting it all together, you can build a robust GraphQL API by rewriting incoming queries into non-redundant ground-typed normal form, computing the expected result size in polynomial time, and then proceeding to execute the query only if the expected size is below an acceptable threshold.
Quick Hits:
Math / Science / Behavior / Economics
Knowing Monads Through The Category Theory
We all know that programming is based on math, what is perhaps not so clear is that knowing more math is going to help us become better developers. And although in all types of programming, math is fundamental, it is even more so in functional programming.
A very widespread concept which I’m sure you have heard lately is the monad. It is a mathematical concept of the category theory applied later in functional programming.
I first heard the term when I started programming in Scala. After much searching on the Internet, I only found articles explained by and for mathematicians or articles that talked about programming and not math. On the other hand, some abuse of language is done to refer to things that are not monads as such. So we will try to talk here, without losing the mathematical rigor, about what the monads are, and then how and why they will help us become better developers. Although they are very abstract and complex terms we will try to do it all in a simple way.
In order to understand the monads, we must first learn about two concepts, categories and functors.
- Screw burritos give me Game of Thrones and Lord of the Rings.
- Seriously though, the map is a really nice way to do this. And probably a more helpful analogy just because of the obvious visual mapping that is often helpful.
Blockchain / Crypto
The Verge Hack, Explained
Cryptocurrency enthusiasts are keen on telling ordinary civilians how safe and secure the Blockchain protocols powering their favorite coins are. Indeed, major cryptocurrencies like Bitcoin and Ethereum have maintained their security quite well — better, arguably, than any other digital asset/payment system in history — which is pretty remarkable, considering that they are unbacked digital money free from any single party’s control with an effective multi-billion dollar bounty on their proverbial heads.
Many, however, will go a step further, and declare said cryptocurrencies to be literally “unhackable.” This is, at the very least, a tactical error, since the proliferation of the “unhackable” meme forces the enthusiast into some awkward positions when and if certain events unfold. Like, say, a hack.
In such an event, it seems that, if nothing else, an explanation is in order.
- What a great write up.
- Beyond the takeaways mentioned at the end let me just say some things that should be obvious or if they aren’t please be aware: distributed programming is very hard. Dealing with time is crazy difficult even when not attempting to program it. It’s only made worse when attempting to coordinate work. All of this while dealing with economics, finance and the worlds of problems that fall out them.
Quick Hits:
- Building Layr: An Experimental Decentralized Cloud Storage System
- $$$: How a few companies are bitcoining it
- The Hidden Predictive Power of Cryptocurrencies: Evidence from US Stock Market
- The Security Token Thesis
- MIT’s Interesting Proposal for a More Stable Financial System
- The Ethereum-blockchain size has exceeded 1TB, and yes, it’s an issue
AI / Machine Learning / Data Science / Statistics
ML beyond Curve Fitting: An Intro to Causal Inference and do-Calculus
You might have come across Judea Pearl’s new book, and a related interview which was widely shared in my social bubble. In the interview, Pearl dismisses most of what we do in ML as curve fitting. While I believe that’s an overstatement (conveniently ignores RL for example), it’s a nice reminder that most productive debates are often triggered by controversial or outright arrogant comments. Calling machine learning alchemy was a great recent example. After reading the article, I decided to look into his famous do-calculus and the topic causal inference once again.
Again, because this happened to me semi-periodically. I first learned do-calculus in a (very unpopular but advanced) undergraduate course Bayesian networks. Since then, I have re-encountered it every 2–3 years in various contexts, but somehow it never really struck a chord. I always just thought “this stuff is difficult and/or impractical” and eventually forgot about it and moved on. I never realized how fundamental this stuff was, until now.
This time around, I think I fully grasped the significance of causal reasoning and I turned into a full-on believer. I know I’m late to the game but I almost think it’s basic hygiene for people working with data and conditional probabilities to understand the basics of this toolkit, and I feel embarrassed for completely ignoring this throughout my career.
In this post I’ll try to explain the basics, and convince you why you should think about this, too. If you work on deep learning, that’s an even better reason to understand this. Pearl’s comments may be unhelpful if interpreted as contrasting deep learning with causal inference. Rather, you should interpret it as highlighting causal inference as a huge, relatively underexplored, application of deep learning. Don’t get discouraged by causal diagrams looking a lot like Bayesian networks (not a coincidence seeing they were both pioneered by Pearl) they don’t compete with, they complement deep learning.
- I love this quote:
it’s a nice reminder that most productive debates are often triggered by controversial or outright arrogant comments
Quick Hits:
- $$$: Toyota takes a winding road to autonomous vehicles
- Human Interpretable Machine Learning (Part 1) — The Need and Importance of Model Interpretation
- A guide to an efficient way to build neural network architectures- Part I: Hyper-parameter selection and tuning for Dense Networks using Hyperas on Fashion-MNIST
- Wide Residual Networks with Interactive Code
- Stitch Fix: The Amazing Use Case Of Using Artificial Intelligence In Fashion Retail
- Brookings survey finds worries over AI impact on jobs and personal privacy, concern U.S. will fall behind China
If you’re looking for help with your architecture or development organization feel free to reach out: realkinetic.com @real_kinetic
You can follow me directly @lyddonb
