"Predictions are hard. Especially when they're about the future." -Yogi Berra

A hobby of mine is to try to understand where things are going. I love knowing ahead of time what's going to happen - aka "the future is already here — it’s just not very evenly distributed."

Inherently I think we all know that there is an advantage to being ahead of the curve, but I like to challenge myself to take it up a notch and document my thinking at reasonable intervals (then review when the time comes up). This particular article is based on a speech I mostly wrote in 2019 and conducted in ~2020. I'm turning the speech into an article at the request of my buddy Patrick Thurmond so forgive any lack of cohesion since they were originally just notes.

(Note: this article is focused very specifically on developer/engineer software tools, frameworks, languages, and systems... meaning it will offer very little for laypeople. I plan to follow up with another article that will touch on larger technology trends that will be influenced by this piece)

1. Setting the stage
2. According to the Experts...
3. General Predictions!
4. Vaporware
5. Tech I'm on the fence about
6. 10 Technologies you need to be aware of

Setting the stage.

Imagine the world back in 2014 ... MVC frameworks (Rails, Laravel, Django) are big, jQuery dominates the web development frontend (framework). "Big Data," "Cloud Computing," and the tools powering distributed computing are all the rage. Node.js is picking up steam, Swift is announced to great fanfare, meanwhile, AWS releases their Functions as a Service product (Lambda) and virtually no one notices. Oculus/ Whatsapp are acquired by Facebook and Satya Nadella becomes Microsoft CEO.

What would safe predictions look like in 2014?

Perhaps Node.js seems like it will be a big deal. Maybe Angular.js will replace jQuery? NoSQL might challenge SQL databases. Mobile is CRAZY hot and compile-to-native mobile frameworks might offer a path for web developers to offer mobile (making mobile even bigger)... and Apache Spark is a must-learn for any serious cloud practitioner.

How safe would those predictions be? I would argue they're pretty "meh.

What is next according to the experts

According to "experts", "venture capitalists", "thought leaders", etc the following are the big blocks that make up the future of technology:

• Artificial Intelligence/ Machine Learning!
• Robotics
• Neurotech (e.g. Nueralink)
• Biotech (e.g. Ginko Bioworks, 23andMe)
• Custom Microprocessors
• Quantum Computing
• New Energy storage
• Space
• IOT & embedded systems
• Health Tech

(Note: if you notice, all of these technologies are expensive and will take a lot to move the needle on)

My 10 Year General Predictions

Note: I believe in having "skin in the game" when I make predictions, so these predictions also often represent my real financial positions. Meaning this isn't hot air and "thought leadership"... these are real bets that I have made wherever possible

1. Voice-controlled applications will significantly grow in use and users (e.g. Alexa, Google, etc)- Why? 1billion people in Southeast Asia have to type in logograms... which is way WAY slower than talking to a smartphone.

2. Unmanned retail and/or unmanned warehouses (dark warehouses) will grow significantly in use. Why? There seems to be a tipping point coming with labor dynamics that no one seems to be appreciating. A raise to the minimum wage could/will force the hand of Entrepreneurs to switch from people to technological solutions. Example: Amazon Specialties.

• Electric vehicles (NOT Fuel Cell) will overtake Internal Combustion vehicles. Why? Fewer moving parts (lower cost of ownership), Quieter, Don't require Gas or Petroleum infrastructure. Plus we're getting very close to a place where economies of scale for EVs equal or overtake ICE vehicles - purchase price, gas, insurance costs for ICE vehicles will start to go up as EV adoption increases. Lastly, there has been a move on the part of institutional investors to pull "Energy Companies" out of their portfolios - this is a non-trivial change in the economic environment of ICE vehicles.
• The Death of Traditional Retail (but NOT all retail) will occur. Why? Large retail spaces with premium rent for prime commercial locations made a lot of sense when consumers were able to get the largest/best selection from these large retail locations (AKA drive from far away to a central car-focused retail center). The unit economics are ceasing to make sense though as ecommerce (Amazon) has better selection with lower prices (and no driving!). I would expect some companies to transition to mixed digital/retail models and/or experience-focused retail experiences but not all companies. Death is probably a strong word because I don't think it will be very "visible", but JC Penny employed more people than the entire coal industry.
• Drone delivery will become a thing. This is HIGHLY dependant on regulation... but I feel like it will come through, the Unit economics make a LOT of sense in urban/suburban environments. I imagine it will be kind of like Waymo at first- restricted to a few cool cities (till 2030) but will be generally available by 2040.
• Ag-Tech will add tools such as precision weeding, ML in agriculture, hydroponics/ Urban/ vertical Farms - Few people outside of agriculture are aware of what GPS did for crop yields. The next set of technologies could have a similar impact, and a nice bonus would be the decrease in pesticides that make it into the ecosystem
• Nano Satellites for the "other 2 billion people" - Elon Musk built perhaps the largest (barely talked about) business that may have ever existed with Starlink. My prediction is that a non-trivial amount of people end up using it for their internet before 2030.
• ImmunoTherapy- though I'm really surprised this hasn't had faster adoption I still feel strongly about treating cancer with immunotherapy. I know a bunch of people that head down to Mexico for Stem Cell and Immunotherapy- so it's started but I'm still genuinely pissed off that it hasn't become a full-fledged offering to Americans right now.

Vaporware

The following is a list of things that I do not believe will materialize.

• Hands down the biggest waste of R&D dollars I can think of are flying cars. Energy isn't cheap enough to warrant adoption, it's an FAA nightmare, but mostly there are a vast number of other technologies easier to implement than flying Cars.
• There's been a lot of hype around this generation's Virtual Reality iteration, but I think it will again be vaporware. Why? It's fighting for dollars in the already heavily invested entertainment segment as opposed to trying to pry dollars away from businesses. The main turning point in my decision making was that John Carramack resigned from his CTO position from Oculus, citing he didn't think VR would end up where he wants it to in his lifetime. That guy is one of the smartest people in computing... and if he thinks it ain't going to happen I trust him.
• "Smart" Cities. (e.g. Google in Toronto; Bill Gates, New Mexico). So billionaires love the idea of smart cities, but they always fail on the premise that a city is primarily a people thing not an infrastructure setup. Personally, I can't help but think about Disney's Epcot Center which was originally supposed to be the "city of the future" until Walt Disney died.
• The Hyperloop project. Not a lot to say about it... but I will say I don't think its much of an improvement over MagLev. projects or High Speed rail - both of which were conceptualized ~2 generations ago and can't even be built as a prototype in the United States.
• Universal Basic Income - I believe this warrants a full-fledged article or series itself, but the TLDR; is that none of our current major forms of civilization are equipped to implement such a thing. I have yet to hear a fiscally conservative means of achieving it, so I will retain my skepticism for at least 20 years.
• Food Service Robotics. Zume Pizza made big waves with it's $300M+ investment (note: since my original speech Zume has actually gone out of business. +1 to me) and there are at least 2+ Robot coffee shops built in the last ~5 years (I've tried one... and it was a great cup of coffee IMO)... and yet none of these things represent novel technological improvements. Zume Pizza specifically could have been built in the Early/Mid 1800s from what I can tell with the technology that existed at that time. Burger flipping Robots, Coffee, Sandwiches, etc are all things that historic automation could handle and yet we chose not to, in lieu of human labor. I'm pretty confident that it will be a while before someone is overtaking Domino's model.

Where I'm On the Fence...

• K8s (Kubernetes) and Docker. As I understand it the Docker/K8s is the cheapest way to do horizontal scaling and distributed computing... but they come with a major cost of HIGH complexity and they require specialized (expensive) engineering talent to keep the software entropy from eating a project. BUT they are backed by Google and Y combinator. Also worth mentioning Docker might not be financially viable with an open source model. Basically, my heart says companies/engineers will want the control but my head says: the major cloud providers provide the infrastructure. (Important note: over the last year I have reversed my opinion about Docker/ K8s. I now believe they fall into the vaporware category since IaaS provides superior financials for the majority of companies.)
• Ruby (Programming Language) - Over the last few years Ruby has seen a major decline in bootcamp programs (of course bootcamps have also seen a decline) and overall language prevalence. It has been the last major language to get infrastructure provisions (AWS, Google, Microsoft, etc) but still has a solid opensource offering - Sidekiq, Rails, Sinatra, Jekyll. Seems like a coin toss in my book on whether the language will prevail.
• Blockchain Technologies. There are a ton of companies playing in this space with a lot of investment. Notable companies include: Brave, Defi, Square Cash, Coinbase... but the Libra Project was basically shut down by congress AND there are HUGE energy consumption considerations- E.G. China stepped in and stopped miners. I feel like the benefits of Blockchain technologies outweigh the negatives... but it's more of a gamble on whether projects can get their energy consumption under control. (2021 Note: I still feel like blockchain technologies are on the fence but I am starting to see governments interested in controlling the technology. I believe this actually bodes well for Blockchain tech but it still has a long way to go I would love to see a useful commercial application start picking up)
• Augmented Reality - AR has more uses than VR, but more importantly I actually believe there are more commercial opportunities for AR technology than VR. With that said, developers working on AR problems are still focused on the consumer markets and I'm not seeing enough movement to remove my skepticism. Of course the hardware is less expensive than VR, and there are more business-related uses for AR that I'm leaning towards higher adoption.

10 Technologies you need to be aware of...

1. Infrastructre as a Service

5 years ago we would be talking about what "Stack" you work in... I think those types of discussions are moving towards infrastructure and tooling. Of course, this is kind of "Duh" since everyone is on AWS already right? Yes and no... companies still seem to be managing a lot of stuff themselves still... which leads me to believe that companies that are willing to outsource these infrastructure tasks will ultimately gain market share. The same goes for Developers/ Engineers that learn how to piece Infrastructure components together.

Why you should care?

It's a massive productivity boost You no longer need to setup your own authentication, configure load-balancers, or split out read/write instances of a database. Someone did it for you and did it better than you. AND it's cheaper - No one seems to like doing the math but DevOps engineers cost a lot fuck ton of money. IaaS is extremely cheap until you are really using it. (note: I am even more bull on IaaS than I was when I wrote this speech... and in a related note the costs for IaaS have come down substantially so I feel like it is no longer an advantage to configure your own infrastructure)

2. Machine Learning (isn't a fad)

Artificial Intelligence and Machine Learning had a moment recently where they became incredibly sexy. Hell, when you can sell a shitty sci-kit learn implementation for a $300 Million valuation... some heads will naturally turn... but underneath the "sexy" exterior there are real solutions that the technology can solve. That doesn't mean it's all kittens and sunshine. In my book there are two completely different AIs at work: the "AI"/"ML" that will do something useful for you and the "AI" the sales guys are selling. The stuff the sales guys are selling is going to go the way of the dodo... but the stuff that helps classify and makes edge cases easier to solve... I believe fully in that technology. Examples of Machine Learning utilities that are incredibly useful right now: recommendation systems, anomaly/ fraud detection, Image/facial recognition systems, Translation systems, etc

Why you should care?

Classical Machine Learning Algorithms (e.g. Scikit Learn), Tensorflow, Pytorch, etc are all going to increase productivity. Sure there is a lot of bullshit hype around "AI" but there's a reality that programming rules that can be inferred by an algorithm is just more efficient than having a person try to code the same thing.

##3. Serverless / FaaS

FaaS are an aspect of IaaS (see above), but I feel that they are owed their own section because they are such a big deal. Functions as a Service offer a unique value proposition that you can just worry about code (business logic) and nothing else. Why does that matter? Because Code is a liability, Software is an asset. Software you own, where you can externalized the maintenence costs, will ultimately provide companies the most value for their money (assuming all other factors are equal). This is why FaaS are SOOOOOOOOOOO powerful. You just need less code. You don't need a framework... you don't need routing, middleware, additional auth systems, etc. You need fewer dependencies... fewer everything really.

Looking for more reasons? If you are doing a serverless stack (FaaS) you don't need (or you need fewer) dedicated devops engineers (who are, as previously mentioned, very expensive). Furthermore, it's effectively infinte scale for very little money.

Also a BIG deal: Cloudflare Workers

Important sidebar - there is one serverless product that seems to stand out against the others based on its performance and cost: "Cloudflare Workers". Cloudflare workers function similarly to any other Serverless/ FaaS product the only difference is that they use V8 isolates rather than containers (Note: which only support JS/ WASM). It turns out that this makes workers 441% faster than AWS (according to smashing mag) and the cheapest FaaS available (3x Cheaper than AWS). Additionally, because Cloudflare's primary offerings are security and CDN related Workers provide the highest availability- lowest latency around (thanks to their 165 data centers)

4. NewSQL

Databases have a problem... actually, they have two problems: 1) There is a LOT of innovating left to do in perhaps the most lucrative area in software ...and in full dramatic irony, 2) SQL is perhaps the stickiest technology ever to have been built in software. Over the last ~10 years there has been an incredible increase in the velocity and deployment of data persistence technologies (Spark/HDFS, NoSQL, Column stores, Graph Databases, key-value stores) and each of these technologies has its own very useful niche. Yet, most developers that I know still default to a good old Relational Database Management System when they build a new project. The NewSQL movement seems to be mostly about this. Databases like CockroachDB are trying to innovate in the data persistence space while making adoption simpler with good ole SQL as their query language.

Why you should care?

(As always, if you're doing a simple CRUD app/API or doing basic analytics just use Postgres - it could be the most boring technology in the world. Which more than likely makes it perfect for your application.) For people/companies that are trying to get the most out of their technology and/or are doing something novel... use the right tool for the job. NewSQL technologies might be that tool. As an example, Cockroach DB provides better availability and automated sharding while still offering the familiarity of SQL for a query language. Column Stores such as Redshift or BigQuery offer a major speed increase for working with big data.

5. SSGs & JAM Stack

So let me start by saying I think JAM stack is the world's lamest name- I in the camp where I don't feel we need a 3 or 4 letter acronym every time we string together a couple of pieces of software.

...but now that we've gotten that out of the way. Everything that was old is new again. The JAM stack and Static Site Generators are effectively a new spin on our old favorite technologies (Javascript, CSS, APIs, and Markup). Put another way, it's a movement to utilize our most basic web technologies as a sort of software minimalism (but with new tooling).

I am a BIG fan of this kind of minimalism, but it turns out I'm not the only one. Several products such as Netlify, Cloudflare Workers Sites, and Firebase have all taken advantage of new technologies to allow developers to easily work in the JAM stack. At the same time the opensource community has embraced these new paradigms by growing the foundational technologies. SSGs like Jekyll, Gatsby, Metalsmith or Eleventy, Frontend frameworks like Vue, React, or Svelte have all become quite mature and are frequently used as the core technologies for modern applications

Why you should care?

JAM stack and SSGs offer Performance improvements over traditional backends, Productivity improvements via simpler deployments but best of all it is Cheaper!

6. Rust

Rust is a low-level, performant language developed and maintained by Mozilla (2021 Note: Rust has been split out into its own foundation and is even being considered as the second language for the Linux Kernal. Quite a lot of moment in the last year). Rust is type-checked, memory safe without garbage collection, and extremely performant. The language places a high value on quality error messages and has a killer ecosystem. It is fast becoming the default choice for low-level development in all sorts of technological areas.

Why should you care?

Well its been Stack Overflow’s most loved language for four years in a row (via the stack overflow survey), it's elegant, but performant and is useful for other stuff that's coming up in the next couple sections of this article.

7. WASM/ Web Assembly

WebAssembly is a new type of code that can be run in modern web browsers and it provides low-level access to the browser/ V8. It is designed to be an effective compilation target for low-level source languages like C, C++, Rust, etc.

(A quick note about performance/ speed: For the bulk of my career I never understood why people cared so much about the speed/ performance of their software. Over the last few years though it has become incredibly clear why it matters. Speed and performance are the ultimate empathy for the users of your software. It takes a great amount of technical skill and forethought to be able to provide performant software, as well you need to care about EVERYONE ... which can be a big challenge at scale. Providing your users a quick, snappy experience is one of many things that will make using your software enjoyable and consequently more useful as a whole. Therefore technologies like WASM are valuable because they make it easier to be empathetic to your users)

Why should you care?

Um... Crazy performance improvements and it adds C, C++, & Rust to the list of languages that can work for the browser.

8. Deno

Deno is a Typescript runtime written in Rust. What is Deno really? It's an improvement and a replacement for Nodejs. Deno offers better security, a standard library, a module system that uses URLs and decreases dependency fanning, and is delivered in a single executable file so it's awesome for Serverless!

Why should you care?

Deno was created by Ryan Dahl and it's going to replace Node and become the default environment for building applications. Period. Full stop.

I could (and might) write a pretty massive article on just this topic, but I am just going to go all cryptic and say you should check this out for yourself.

9. Entropic

(2021 Note: I took a little bit of a risk when I threw Entropic on this list and it turned out that I have to eat crow on this one. C J Silverio has been a big deal in open source for a long time and knows how to publish. Unfortunately, it looks like my intuition failed and the project was eaten by entropy)

Entropic is a federated package registry and CLI created by C J Silverio previously of the NPM company. The federated part is the interesting part- It mirrors all packages that users install from the legacy package manager - meaning there can be multiple copies out there but a single source of truth.

Why you should care?

Entropic offers an improvement over NPM and YARN as removes control from central entities while maintaining secure transport of packages. In my opinion, the really big deal about this is that there is a movement towards taking back the web from the mega-corps.

10. Apache Arrow

Apache Arrow is a column memory format organized for efficient analytics operations on modern hardware.

I was previously a little "meh" about the project until learned that Wes McKinney is the original author and leading development efforts. Wes is a prolific software developer both in the commercial and engineering sense (the guy builds great software and great companies)- so that carries some weight with me. Of course, one developer wouldn't validate a technology by themselves, but as I started digging I began to see the problem that they are solving is somewhat akin to the reason why Pandas exists, but the approach is lower level and very much needed for the future of data analytics.

Why you should care?

If you are doing data analytics and/or need to build a proof of concept for something that you would love to do in memory... Arrow might make a big difference in your life coming up.

Closing thoughts

The above technologies are my best guess for how things will shake out. I think its worth noting that I believe there is an underlying theme to all technological movements (aka the things that make up the future):

1. The technology is actually superior in a way that makes a difference over the status quo. (conceptually related to "the innovator's dilemma")
2. There is usually a voice or a narrative that helps the rest of us move forward.
3. There are people involved that we are willing/wanting to put our faith in.

...with that said, even if a technology has these aspects, it is far from a guarantee that it will proliferate. There are many externalities that factor in and they are impossible to predict. So in the least, be aware. These technologies might the future, or they might show you "what not to do" so you can better predict the future for yourself.