Overview: Innominds recently hosted a dinner series where a number of industry leaders from big data joined together to discuss the impact of big data; it’s shortcomings and to lay a future roadmap on how product companies should structure their big data solution.
Speaker - Rao Yendluri, CTO iFusion Analytics, an Innominds Company, San Jose
With more than 30+ years of experience in the computer software industry and a proven track record in building geographically distributed R&D organizations and delivering distributed database and data warehouse products, Rao Yendluri is one of the brightest minds in the field of Big Data Analytics. An alumni of BITS Pilani, Rao has had a stellar career as a tech leader/thinker and has served in multiple positions over several decades.
Rao Yendluri, CTO, iFusion Analytics, acted as a moderator for the panel discussion on IBDaaS (integrated Big Data as a Service) to accelerate insights into big data. His talk aimed at defining the term IBDaaS so that it acts as a way to accelerate adoption of big data as a service and get insights into data faster.
The Value of IBDaaS
Rao explained how a typical cloud stack would be with other online services for cloud computing like IaaS (Infrastructure as a service), PaaS (Platform as a Service), SaaS (Solution as a Service) and BDaaS (Big Data as a Service). In most of the cases, IaaS focuses on infrastructure optimization which is providing hardware, network, storage, and OS. PaaS focuses on Hadoop and other database services. IaaS and PaaS focus on Hadoop and optimized infrastructure for performance. PaaS and SaaS focus on Hadoop and features for productivity and exchangeable infrastructure. A combined stack of IaaS, PaaS and SaaS focus on complete vertical integration for features and performance With IaaS, PaaS and SaaS as a stack, users still have to write their own application or business solution and provide it as a service.
With regards to IaaS, Rao mentioned that the popular providers are Azure, Google and Amazon. There also a few private providers. The services we get from them includes Compute instances, Hardware, Storage and Network.
When it comes to PaaS, it could be Postgres, Oracle, Hadoop, SQL and NoSQL Datastore such as Cassandra with services on top of it. If it is a database, then we get a bunch of APIs' so that the user can build an application. Of late, many offering Big Data as a Service (BDaaS) where BDaaS refers to Hadoop as a service with optimized IaaS.
Even with BDaaS, users still have to write build their own solution and make it as a service for its user community. These solutions (Healthcare, Insurance, Finance, Banking, IoT, IIoT, etc.) (aka Vertical Solutions) take time and resources to build. That makes ROI in big data is disappointing. What we need to improve ROI in Big Data is vertical solutions integrated with BDaaS and configurable for each user. This will improve ROI in Big Data and provides faster insights into Big Data.
Thus, what we need is a fully vertically integrated BDaaS that combines the performance and feature benefits of BDaaS.
With regards to the performance of BDaaS, Rao said users are deploying s not 1 node or 10 nodes but 1000s' of nodes. So the question we need to ask is:
"How do you optimize it so that 1000 nodes can work together and scale for volume, velocity and variety?”
Here, Rao emphasized on the need for optimization. He said that it is not just about taking sandbox from Hortonworks, installing it and making it over one node. The challenge here is, when you are dealing with a Petabyte, will it work? Rao said that this needs a lot of fine tuning.
He then spoke about optimised infrastructure which includes optimised storage, optimised cluster, optimised volume and optimised network. Rao said that we need to combine all of this in order to get a scalable solution.
Also, when it comes to building custom applications, the issue is, somebody has to still write it. The challenge lies in getting the solution sooner into the customer’s hands so that all they have to do then is to give data to find the solution. He noted that when you take a pluggable infrastructure with featured big data and start adding features on top of the core BDaaS, the customer still has to go ahead and build it. Therefore, the question is beyond common Hadoop ecosystem.
Why is IBDaaS special?
Speaking on the unique capabilities of IBDaaS, Rao explained that with IBDaaS, we get core big data which is optimized for network, storage, and more importantly, it gives specific vertical solutions.
For example if you prefer Amazon, you will have one or more AMIs' and when you click the Amazon instances, the cluster is ready for that solution. All the customer has to do then is to feed the data. The service performs all the scalable activities and cleaning/preparing the data for applying the algorithm. Whether it is an anomaly detection algorithm, recommendation engine or something similar, the solution is ready for the customer for that particular domain or vertical.
This is because, anomaly detection can be used in multiple ways like detecting anomalies in network security, healthcare, power turbines, HVAC, Pharma Compressors etc.
This makes IBDaaS special and valuable to customers, Rao concluded.