Welcome!

Apache Authors: Liz McMillan, William Schmarzo, Christopher Harrold, Elizabeth White, Talend Inc.

Blog Feed Post

Ryan Sheftel: "R on the Trading Desk"

by Joseph Rickert In a post last week, I offered some first impressions about R/Finance 2013. Apparently, I was way off in estimating that 30% of the attendees were academics. The R/Finance organizers were quick to point out that percentage of academics attending the conference has been a constant 10% over the years; and this year was no different. Why is this important? Well, first off it points to the level of sophistication of the industry attendees who came to hear talks that were mostly very technical, both with respect to the level of mathematics involved and R usage. And, perhaps it explains why there was very little hype at the conference. There were no exagerated claims for Hadoop or any other technology, and discussions involving big data were very "matter of fact". When it comes to technology, the quants are a sober lot, pragmatic, competent and comfortable with the latest technology trends. The opening keynote presentation: R on the trading Desk (PDF) by Ryan Sheftel, Managing Director of Fixed Income Division at Credit Suisse, underscored this last point. Ryan, an engaging and experienced speaker, provided some extraordinary insight into the stance quants and traders take toward technology: By way of background, Ryan noted that fixed income markets which provide immediate liquidity between buyers and sellers are decentralized. Traders are constantly taking on risk which they try to mitigate by predicting what their clients are going to do. Ryan noted that machine learning ideas, very often implemented in R, have been a “great boon” to the industry. Before the financial crisis, the quants were focused on building complex models to price options, now they are focused on prediction using time series models. These days, the technology on a fixed income desk at Credit Suisse includes: SQL and data sources such as OneTick R and time series libraries such as zoo and xts R libraries for reshaping and manipulating data RStudio’s integrated development environment This combination of technology has helped to “lower the bar to accessibility” meaning timely and high quality models. More people are answering their own questions, testing their own code and doing version control. As a consequence of this, the barrier between quants and traders is breaking down. Both groups are using the same tools and, if I understood Ryan correctly, there are even expectations that traders will do their own unit testing! The idea is to make quality assurance part of the creative process. It is expected that more lines of code will be devoted to testing than calculations and that QA is the responsibility of the person who had the idea. Ryan noted that this way of working and the kinds of tools available are affecting the kinds of people being hired. Although Ryan never used the term, it is clear that at least one organization within Credit Suisse is building its future with "data scientists". Ryan’s talk covered much more ground than I can explore here; much more even than his slides indicate. However, I would be remiss not to mention Ryan's challenges for R and his warning against complacency. R is apparently deeply entrenched at Credit Suisse, It is now an “acceptable” tool at the bank and quants and traders write their own proprietary packages. However, a disaster linking R to a big loss, such as the Excel error that apparently contributed to the London whale debacle, could dislodge it. In Ryan’s opinion it is just "too easy to do some things" in R. Ryan stated that programming rigor around the language and better error tracking would be very useful here. This is the classic tradeoff between freedom and security, between providing individual users with powerful feature-rich tools and limiting the damage a careless individual can cause. These kinds of concerns help to make the case that some organizations could benefit from a managed distribution of the R language. As for the warning against complacency: Ryan noted that Python with the Pandas library also has a place on the trading desk. All of the presentations for R/Finance 2013 are online here. Many thanks to the speakers and the conference organizers for making them available. Many of the presentations have snippets of R code that are helpful following the math. (See for example the presentations of Bernhard Pfaff and Sanjiv Das.) For those of you who still want more; the cumulative collection of R/Finance presentations is becoming quite a online library. Have a look at the presentations from previous years: R/Finance 2009 R/Finance 2010 R/Finance 2011 R/Finance 2012 

Read the original blog entry...

More Stories By David Smith

David Smith is Vice President of Marketing and Community at Revolution Analytics. He has a long history with the R and statistics communities. After graduating with a degree in Statistics from the University of Adelaide, South Australia, he spent four years researching statistical methodology at Lancaster University in the United Kingdom, where he also developed a number of packages for the S-PLUS statistical modeling environment. He continued his association with S-PLUS at Insightful (now TIBCO Spotfire) overseeing the product management of S-PLUS and other statistical and data mining products.<

David smith is the co-author (with Bill Venables) of the popular tutorial manual, An Introduction to R, and one of the originating developers of the ESS: Emacs Speaks Statistics project. Today, he leads marketing for REvolution R, supports R communities worldwide, and is responsible for the Revolutions blog. Prior to joining Revolution Analytics, he served as vice president of product management at Zynchros, Inc. Follow him on twitter at @RevoDavid

@ThingsExpo Stories
IoT is rapidly changing the way enterprises are using data to improve business decision-making. In order to derive business value, organizations must unlock insights from the data gathered and then act on these. In their session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, and Peter Shashkin, Head of Development Department at EastBanc Technologies, discussed how one organization leveraged IoT, cloud technology and data analysis to improve customer experiences and effi...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform and how we integrate our thinking to solve complicated problems. In his session at 19th Cloud Expo, Craig Sproule, CEO of Metavine, will demonstrate how to move beyond today's coding paradigm ...
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 19th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo Silicon Valley Call for Papers is now open.
Big Data engines are powering a lot of service businesses right now. Data is collected from users from wearable technologies, web behaviors, purchase behavior as well as several arbitrary data points we’d never think of. The demand for faster and bigger engines to crunch and serve up the data to services is growing exponentially. You see a LOT of correlation between “Cloud” and “Big Data” but on Big Data and “Hybrid,” where hybrid hosting is the sanest approach to the Big Data Infrastructure pro...
"My role is working with customers, helping them go through this digital transformation. I spend a lot of time talking to banks, big industries, manufacturers working through how they are integrating and transforming their IT platforms and moving them forward," explained William Morrish, General Manager Product Sales at Interoute, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
A critical component of any IoT project is what to do with all the data being generated. This data needs to be captured, processed, structured, and stored in a way to facilitate different kinds of queries. Traditional data warehouse and analytical systems are mature technologies that can be used to handle certain kinds of queries, but they are not always well suited to many problems, particularly when there is a need for real-time insights.
With 15% of enterprises adopting a hybrid IT strategy, you need to set a plan to integrate hybrid cloud throughout your infrastructure. In his session at 18th Cloud Expo, Steven Dreher, Director of Solutions Architecture at Green House Data, discussed how to plan for shifting resource requirements, overcome challenges, and implement hybrid IT alongside your existing data center assets. Highlights included anticipating workload, cost and resource calculations, integrating services on both sides...
"We are a well-established player in the application life cycle management market and we also have a very strong version control product," stated Flint Brenton, CEO of CollabNet,, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Unless your company can spend a lot of money on new technology, re-engineering your environment and hiring a comprehensive cybersecurity team, you will most likely move to the cloud or seek external service partnerships. In his session at 18th Cloud Expo, Darren Guccione, CEO of Keeper Security, revealed what you need to know when it comes to encryption in the cloud.
We're entering the post-smartphone era, where wearable gadgets from watches and fitness bands to glasses and health aids will power the next technological revolution. With mass adoption of wearable devices comes a new data ecosystem that must be protected. Wearables open new pathways that facilitate the tracking, sharing and storing of consumers’ personal health, location and daily activity data. Consumers have some idea of the data these devices capture, but most don’t realize how revealing and...
What are the successful IoT innovations from emerging markets? What are the unique challenges and opportunities from these markets? How did the constraints in connectivity among others lead to groundbreaking insights? In her session at @ThingsExpo, Carmen Feliciano, a Principal at AMDG, will answer all these questions and share how you can apply IoT best practices and frameworks from the emerging markets to your own business.
Ask someone to architect an Internet of Things (IoT) solution and you are guaranteed to see a reference to the cloud. This would lead you to believe that IoT requires the cloud to exist. However, there are many IoT use cases where the cloud is not feasible or desirable. In his session at @ThingsExpo, Dave McCarthy, Director of Products at Bsquare Corporation, will discuss the strategies that exist to extend intelligence directly to IoT devices and sensors, freeing them from the constraints of ...
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
Extracting business value from Internet of Things (IoT) data doesn’t happen overnight. There are several requirements that must be satisfied, including IoT device enablement, data analysis, real-time detection of complex events and automated orchestration of actions. Unfortunately, too many companies fall short in achieving their business goals by implementing incomplete solutions or not focusing on tangible use cases. In his general session at @ThingsExpo, Dave McCarthy, Director of Products...
Traditional IT, great for stable systems of record, is struggling to cope with newer, agile systems of engagement requirements coming straight from the business. In his session at 18th Cloud Expo, William Morrish, General Manager of Product Sales at Interoute, outlined ways of exploiting new architectures to enable both systems and building them to support your existing platforms, with an eye for the future. Technologies such as Docker and the hyper-convergence of computing, networking and sto...
WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, discussed the importance of WebRTC and how it enables companies to focus...
With an estimated 50 billion devices connected to the Internet by 2020, several industries will begin to expand their capabilities for retaining end point data at the edge to better utilize the range of data types and sheer volume of M2M data generated by the Internet of Things. In his session at @ThingsExpo, Don DeLoach, CEO and President of Infobright, discussed the infrastructures businesses will need to implement to handle this explosion of data by providing specific use cases for filterin...
IoT generates lots of temporal data. But how do you unlock its value? You need to discover patterns that are repeatable in vast quantities of data, understand their meaning, and implement scalable monitoring across multiple data streams in order to monetize the discoveries and insights. Motif discovery and deep learning platforms are emerging to visualize sensor data, to search for patterns and to build application that can monitor real time streams efficiently. In his session at @ThingsExpo, ...
Early adopters of IoT viewed it mainly as a different term for machine-to-machine connectivity or M2M. This is understandable since a prerequisite for any IoT solution is the ability to collect and aggregate device data, which is most often presented in a dashboard. The problem is that viewing data in a dashboard requires a human to interpret the results and take manual action, which doesn’t scale to the needs of IoT.
Internet of @ThingsExpo has announced today that Chris Matthieu has been named tech chair of Internet of @ThingsExpo 2016 Silicon Valley. The 6thInternet of @ThingsExpo will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.