Click here to close now.

Welcome!

Apache Authors: Roger Strukhoff, Elizabeth White, AppDynamics Blog, Liz McMillan, Carmen Gonzalez

Related Topics: BigDataExpo® Blog, JAVA IoT, Microservices Expo, Microsoft Cloud, Containers Expo Blog, CloudExpo® Blog, Apache

BigDataExpo® Blog: Blog Feed Post

Big Data | Thinking Outside the Firewall

Big Data is a by-product of the Internet and the ever increasing power of computers

A few months back, Gartner placed big data at the peak of its hype cycle for cloud computing, meaning most big data products are solutions looking for a problem. I always find this bad entrepreneurial habit to be one of the most frustrating of our industry. Having recently joined Meltwater as head of marketing and product (BTW Meltwater is hiring marketing and product managers!), I think a lot about big data and how to unleash it’s value to solve important business problems, because that is our business. How does big data go from “so what” to “must have”?

The Big Data Challenge
Big data is a by-product of the Internet and the ever increasing power of computers. Kind of like petroleum sludge. We know there must be great value buried within this vast, raw resource, but the challenge lies in figuring out how to turn it into something useful like plastic, or the other thousands of petroleum products that we produce from the 20% of crude oil that can’t be turned into fuel.

big data by product

This is no small feat. I can confidently predict that there will be no shortage of well-intentioned, well-funded start-ups that fail to live up to this challenge, producing varying versions of gift-wrapped sludge that never quite deliver on the promises of their pitches. Overcoming the hype and producing real value from big data requires much more than data-processing infrastructure. It requires a laser-like focus on creating order-of-magnitude improvements to how we work and live.

Reengineering Across the Firewall
More than a year ago, McKinsey and Company predicted that big data would be “The Next Frontier of Innovation, Competition and Productivity.” Now, I’m not generally one to argue with the likes of McKinsey, especially in this case as I happen to agree with it. If you have the time, I highly recommend checking out the report. At 156 pages, however, it can be a little hard to digest, so I thought I’d fearlessly attempt to boil it down to a blog post by sharinig a little of how we think about the Big Data Challenge @Meltwater.

big data meltwater reengineering

Big data implies a shift in real-time access to valuable information outside the firewall. It offers the opportunity to reengineer business processes that cross the firewalland that benefit greatly from this information, such as competitive strategy, sales, customer support, vendor management, employee recruiting, etc.

Cloud-based businesses create value in one of two ways: lowering TCO (cost advantage) and network-enabled innovation (differentiation). Most early-entry, enterprise SaaS applications like Salesforce.com are really not that different from their on-premise counterparts in feature and function. They rely on lower TCO as the primary driver of adoption, and while they expand the market through lower prices to SMBs, they are locked into a replacement battle against on-premise software leaders like Oracle, SAP and Microsoft. However, there are a handful of truly innovative enterprise SaaS and cloud categories that mine the Internet for all it’s worth and have no on-premise equivalent, such as search, media monitoring, marketing automation, Web analytics, social media marketing, human capital management, and cloud integration. These categories all have one thing in common, they reengineer business processes that cross the firewall by leveraging data outside the firewall.

Reengineering is a business term that gained popularity in the early nineties as client-server washed away mainframe applications en masse. The idea was to leverage the new technology to redesign business processes for dramatic gains in productivity, as opposed to just upgrading legacy systems. Fast-forward 20 years and we find ourselves in a similar situation. We’ve spent billions of dollars on ERP, CRM, BI and countless other software acronyms to automate every last internal business process. Inside the firewall, there is very little left to do.

The vision of big data should not be an upgrade, like the next generation of enterprise business intelligence, only bigger. Big data is a fundamental shift in real-time access to valuable information outside the firewall. It offers the opportunity to reengineer business processes that cross the firewall and that benefit greatly from this information, such as competitive strategy, sales, customer support, vendor management, employee recruiting, etc. For example, it is one of the great ironies of enterprise software that in most companies customers never touch the customer relationship management system.

Social Business and Big Data
If any emerging category can claim more hype than big data, it is social business. This is no accident. Social business and big data are inexorably linked. Big social data empowers social business. Take the case of the social community manager, a new and evolving social business role. The social community manager must engage in a dialog with community members that is personal and relevant. Yet at the same time, the social community manager must sift through millions of online conversations to zero in on specific opportunities for personalized social engagement. Enter big data. Let’s redraw the above diagram for the social community manager.

social community management meltwater

The social community manager needs a social mission control panel to digest the vast amounts of big social data and zero in on the conversations, channels, and community members that require immediate attention. (At Meltwater, we like to call this Meltwater Buzz 3.0 which launched today! ;) )

The social community manager is engaged in the business process of building a social community. To accomplish this daunting task, the social community manager must reach across the firewall to crunch a lot of big data and engage community members in in real-time. The process goes something like this:

  1. Gather big data outside the firewall about the social community
  2. Develop a social campaign strategy informed by insights from the data
  3. Create a social campaign plan of execution
  4. Reach across the firewall to engage the social community
  5. Rinse and repeat in real-time

I think social community management is a lot like air traffic control, including the potential for social media disasters when information, systems or social community managers are not up to the task. The social community manager must digest vast amounts of big data to find the one conversation or one community member that requires immediate attention. The social community manager doesn’t need raw big data. The social community manager needs a social mission control panel to digest the vast amounts of big social data and zero in on the conversations, channels, and community members that require immediate attention.

Big Data and The Cloud | A Match Made in Heaven
The fact that big data originates largely as a by product of the Internet, and the fact that it is, well, big, lead to the natural conclusion that like big data itself, big data-based solutions are best situated in the cloud. It will be the rare global 500 company that is both big enough and motivated enough to house and sift through the mountains of data available out there to build on-premise big data analytics and automation. Economies-of-scale will rule in the aggregation, enrichment and processing of big data with most businesses interested in paying only for results, i.e., insights that can be used to reengineer business processes across the firewall for order-of-magnitude improvements in productivity and service.

Read the original blog entry...

More Stories By Joel York

Joel York is an Internet software executive and popular SaaS / Cloud blogger at Chaotic Flow and Cloud Ave. He is well known for his work in SaaS / cloud business models, sales and marketing strategy, and financial metrics. Professionally, he has managed global sales and marketing organizations serving over 50 countries, including local offices in the United States, United Kingdom, Germany, and India. He holds degrees in physics from Caltech and Cornell and received his MBA from the University of Chicago. Joel York is currently VP Marketing at Meltwater Group and Principal at the Internet startup consulting firm affinitos.

@ThingsExpo Stories
SYS-CON Events announced today that BMC will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. BMC delivers software solutions that help IT transform digital enterprises for the ultimate competitive business advantage. BMC has worked with thousands of leading companies to create and deliver powerful IT management services. From mainframe to cloud to mobile, BMC pairs high-speed digital innovation with robust IT industrialization – allowing customers to provide amazing user experiences with optimized IT per...
The Internet of Things is not new. Historically, smart businesses have used its basic concept of leveraging data to drive better decision making and have capitalized on those insights to realize additional revenue opportunities. So, what has changed to make the Internet of Things one of the hottest topics in tech? In his session at @ThingsExpo, Chris Gray, Director, Embedded and Internet of Things, discussed the underlying factors that are driving the economics of intelligent systems. Discover how hardware commoditization, the ubiquitous nature of connectivity, and the emergence of Big Data a...
SYS-CON Events announced today that MetraTech, now part of Ericsson, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Ericsson is the driving force behind the Networked Society- a world leader in communications infrastructure, software and services. Some 40% of the world’s mobile traffic runs through networks Ericsson has supplied, serving more than 2.5 billion subscribers.
The world is at a tipping point where the technology, the device and global adoption are converging to such a point that we will see an explosion of a world where smartphone devices not only allow us to talk to each other, but allow for communication between everything – serving as a central hub from which we control our world – MediaTek is at the heart of both driving this and allowing the markets to drive this reality forward themselves. The next wave of consumer gadgets is here – smart, connected, and small. If your ambitions are big, so are ours. In his session at @ThingsExpo, Jack Hu, D...
The 4th International Internet of @ThingsExpo, co-located with the 17th International Cloud Expo - to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA - announces that its Call for Papers is open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
SYS-CON Events announced today that DragonGlass, an enterprise search platform, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. After eleven years of designing and building custom applications, OpenCrowd has launched DragonGlass, a cloud-based platform that enables the development of search-based applications. These are a new breed of applications that utilize a search index as their backbone for data retrieval. They can easily adapt to new data sets and provide access to both structured and unstruc...
We’re entering a new era of computing technology that many are calling the Internet of Things (IoT). Machine to machine, machine to infrastructure, machine to environment, the Internet of Everything, the Internet of Intelligent Things, intelligent systems – call it what you want, but it’s happening, and its potential is huge. IoT is comprised of smart machines interacting and communicating with other machines, objects, environments and infrastructures. As a result, huge volumes of data are being generated, and that data is being processed into useful actions that can “command and control” thi...
As the Internet of Things unfolds, mobile and wearable devices are blurring the line between physical and digital, integrating ever more closely with our interests, our routines, our daily lives. Contextual computing and smart, sensor-equipped spaces bring the potential to walk through a world that recognizes us and responds accordingly. We become continuous transmitters and receivers of data. In his session at @ThingsExpo, Andrew Bolwell, Director of Innovation for HP's Printing and Personal Systems Group, discussed how key attributes of mobile technology – touch input, sensors, social, and ...
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades. With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo, June 9-11, 2015, at the Javits Center in New York City. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be
The Internet of Things is not only adding billions of sensors and billions of terabytes to the Internet. It is also forcing a fundamental change in the way we envision Information Technology. For the first time, more data is being created by devices at the edge of the Internet rather than from centralized systems. What does this mean for today's IT professional? In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists will addresses this very serious issue of profound change in the industry.
WebRTC defines no default signaling protocol, causing fragmentation between WebRTC silos. SIP and XMPP provide possibilities, but come with considerable complexity and are not designed for use in a web environment. In his session at @ThingsExpo, Matthew Hodgson, technical co-founder of the Matrix.org, discussed how Matrix is a new non-profit Open Source Project that defines both a new HTTP-based standard for VoIP & IM signaling and provides reference implementations.
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists will peel away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud environment, and we must architect and code accordingly. At the very least, you'll have no problem fil...
Almost everyone sees the potential of Internet of Things but how can businesses truly unlock that potential. The key will be in the ability to discover business insight in the midst of an ocean of Big Data generated from billions of embedded devices via Systems of Discover. Businesses will also need to ensure that they can sustain that insight by leveraging the cloud for global reach, scale and elasticity.
"People are a lot more knowledgeable about APIs now. There are two types of people who work with APIs - IT people who want to use APIs for something internal and the product managers who want to do something outside APIs for people to connect to them," explained Roberto Medrano, Executive Vice President at SOA Software, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect at GE, and Ibrahim Gokcen, who leads GE's advanced IoT analytics, focused on the Internet of Things / Industrial Internet and how to make it operational for business end-users. Learn about the challenges posed by machine and sensor data and how to marry it with enterprise data. They also discussed the tips and tricks to provide the Industrial Internet as an end-user consumable service using Big Data Analytics and Industrial Cloud.
Building low-cost wearable devices can enhance the quality of our lives. In his session at Internet of @ThingsExpo, Sai Yamanoor, Embedded Software Engineer at Altschool, provided an example of putting together a small keychain within a $50 budget that educates the user about the air quality in their surroundings. He also provided examples such as building a wearable device that provides transit or recreational information. He then reviewed the resources available to build wearable devices at home including open source hardware, the raw materials required and the options available to power s...
How do APIs and IoT relate? The answer is not as simple as merely adding an API on top of a dumb device, but rather about understanding the architectural patterns for implementing an IoT fabric. There are typically two or three trends: Exposing the device to a management framework Exposing that management framework to a business centric logic Exposing that business layer and data to end users. This last trend is the IoT stack, which involves a new shift in the separation of what stuff happens, where data lives and where the interface lies. For instance, it's a mix of architectural styles ...
We certainly live in interesting technological times. And no more interesting than the current competing IoT standards for connectivity. Various standards bodies, approaches, and ecosystems are vying for mindshare and positioning for a competitive edge. It is clear that when the dust settles, we will have new protocols, evolved protocols, that will change the way we interact with devices and infrastructure. We will also have evolved web protocols, like HTTP/2, that will be changing the very core of our infrastructures. At the same time, we have old approaches made new again like micro-services...
Connected devices and the Internet of Things are getting significant momentum in 2014. In his session at Internet of @ThingsExpo, Jim Hunter, Chief Scientist & Technology Evangelist at Greenwave Systems, examined three key elements that together will drive mass adoption of the IoT before the end of 2015. The first element is the recent advent of robust open source protocols (like AllJoyn and WebRTC) that facilitate M2M communication. The second is broad availability of flexible, cost-effective storage designed to handle the massive surge in back-end data in a world where timely analytics is e...
Collecting data in the field and configuring multitudes of unique devices is a time-consuming, labor-intensive process that can stretch IT resources. Horan & Bird [H&B], Australia’s fifth-largest Solar Panel Installer, wanted to automate sensor data collection and monitoring from its solar panels and integrate the data with its business and marketing systems. After data was collected and structured, two major areas needed to be addressed: improving developer workflows and extending access to a business application to multiple users (multi-tenancy). Docker, a container technology, was used to ...