Pests, Cake or a Jolly Good Picture of The Future of Software

Pest Control

I have a rather mundane need of finding a pest control service that's located somewhere around the my place. Why somewhere around my place? Because I'd have them do their work early in the morning and my experience is, those who are located far away tend to not show up that early. So, proximity is important.

With that goal, I picked up my android phone, unlocked it, tapped on the blue mic and commanded "pest control in ohsoawesomeville". Google faithfully showed a listing of two pest control services located in ohsoawesomeville, of which one is closed right now. The listing is followed by the traditional blue links to other listings services such as JustDial. For want of a choice, I ended up in Just Dials's landing page, only to find that just one result is from ohsoawesomeville, rest being from farther parts of the city. What's amazing is that JD actually has the data on many many pest control services located in ohsoawesomeville, services they chose to not show on top in its default ranking.

Google listing (top), Default JustDial listing (left) and JustDial listing sorted by location (right)

Getting a search ranking right is hard.

I'd rather have a good listing splashed on my phone screen in response to my voice command, than click on a blue link and then find my way through a broken site search.

How can this be better?

An open Market for Data

JustDial has plenty of data. Unfortunately, they can't create the kind of experience that delights me. Google being the point of entry to the world of information, can control my experience in a good way. It also has better algorithms that understand my intent and get the ranking right. It just didn't have enough data to show.

Data can unlock many possibilities, some are way beyond what its owners can imagine or build. However, owners of the data must be able to monetize the data too. A listings service like JustDial monetizes its data when users visit their website and click on sponsored links or ads. To truly liberate the data, those who have it should be able to monetize it by sharing it with others, without having to build an app or a website.

Now imagine an open marketplace for data-- a platform that connects data providers with thousands of application developers worldwide. A provider connects the data tap to the platform, provides some metadata (e.g. description, license, schema, timestamp, update frequency), perhaps offers a free preview and quotes a price. The platform allows developers to discover a relevant data tap, play with a sample for free, pay and get full access. A platform such as this can enable developers to write myriad data-driven applications and share profits with the data providers.

The data providers or aggregators already exist as businesses. However, the data they provide is often too raw to be used by a large number of application developers. In addition to the marketplace, there needs to be secondary data providers to complete the data eco system. The secondary providers collect raw data and sanitize it by cleansing and de-duplicating.

In-app Data

Data providers collect data using various means, ranging from visiting physical locations to scanning bills, scraping websites, and so on. However, another interesting source of data is the data an application, a mobile app or a website, collects for itself. Owners of the application often view this data as a competitive advantage and would not let others peek into it for anything.

This situation is similar to the era when software companies viewed open source with suspicion and opening up its own source code was unthinkable. Over time, it became apparent that benefits of an open ecosystem outweighs the competitive advantage of a certain clever hack someone implemented in source. I hope a similar dynamic kicks in with data too.

In-app data is often about the users and there are concerns about privacy that is sometimes sited as the reason companies don't open up this data. However, it is possible to anonymize the data to safe guard the users, as Google did with its search logs. It is important to understand that user's privacy needs to be protected not only from the people outside the company but also from the employees too and that means that one should anonymize or encrypt sensitive data anyway.

About the cake

As we go about living our life, we solve one problem after another, mostly without even realizing. Often enough, problems are naturally connected with each other. Like the problem of cutting a cake and eating it. They are different problems, solved with different tools-- a knife and a spoon, respectively. Yet, after we've cut the cake, we expect the spoons are right there so that we can simply start eating.

There are many software applications that are great solutions for specific problems. Yet, quite often, they don't create the more natural and seamless experience together. My banks website allows me to purchase stuff by redeeming bonus points, my wallet application allows me to make payments in an e-commerce site, and a website like policy bazar allows me to invest in retirement savings. Yet, there's no easy way for me to find out if I could have redeemed the bonus points rather than using the cash from my wallet to buy an all-in-one printer, or, if I'm saving the right amount for my retirement, given my current level of spendings.

Immersive Applications

Now imagine an immersive application that understands a certain aspect of my life and is responsible for creating the best possible experience for me. It creates this experience by blending the specific solutions that are now API rather than applications themselves. For example, an imaginary personal finance application uses the API exposed by the bank, the wallet and the investment search engine to create the complete financial management experience for an user. The specific solutions can now focus fearlessly on solving the problems in the best possible ways without worrying about the user experience.

Immersive applications exist even today. For example, consider Uber and paytm integration. Uber helps us find a cab. It also integrates nicely with paytm wallet to solve a connected problem-- that of paying for the cab.

The Jolly Good Picture

Now I'll put these thoughts together in a jolly good picture.

Various layers of a software application of future

I find this picture pretty exciting. Some of the building blocks of the architecture already exist today while others are, hopefully, evolving fast.

See you tomorrow!

Update 27-Nov-2016
Another way to understand this picture is that there are basically 3 different layers in a software application: data, algorithm and user interface. It's best if they remain loosely coupled with each other so that the best algorithms benefit from the best data available and are delivered to humans via the best possible user interfaces.

The application or the user interface layer is evolving in interesting ways too: from desktop applications to websites to smartphone apps and now towards some code that responds to voice in an Amazon Echo , or, to heartbeats in a smart wearable or, perhaps, to facial expressions, in a yet-to-be-invented device.