Data Systems are about exposing data, Services are about hiding it.
This transcribed talk explores a range of data platforms through a lens of basic hardware and software tradeoffs.
A detailed look at the interesting LSM file organisation seen in BigTable, Cassandra and most recently MongoDB
A lighthearted look at Oracle & Google using a metaphorical format. The style won’t suit everyone, but it’s a bit of fun!
- StrataSA-2016: The Data Dichotomy
- Confluent-2017: Microservices with Kafka Series
- QCon-2017: the Power of the Log (video)
- Codemesh-2016: Streaming, Databases & Distributed Systems – Bridging the Divide
- JAXLondon-2016: Microservices for a Streaming World (The Data Dichotomy)
- QCon-2016: Microservices for a Streaming World (video)
- CodeMesh-2015: Contemporary Approaches to Data at Scale (video)
- Øredev-2015: The Future of Data Technology (video)
- JAXLondon-2015: Intuitions for Scaling Data-Centric Architectures (video)
- ProgsCon/JAXF-2015: Elements of Scale
- RBS-2014: Scaling Data
- BigDataCon-2013: The Return of Big Iron?
- JAX-2013: The Return of Big Iron?
- QCon-2012: Where Big Data meets Big Database (video)
- QCon-2012: Progressive Architectures at RBS (video)
- JavaOne-2011: Balancing Replication and Partitioning in a Distributed Java Database
- QCon-2011: Beyond the Data Grid (video)
- UCL-2011: A Paradigm Shift: The Increasing Dominance of Memory-Oriented Solutions for High Performance Data Access
- CoSIG-2011: Oracle Coherence Implementation Patterns (Special Interest Group)
- ICST-2011: Test-Oriented Languages: a new era?
- ICST-2011: Enabling Development Practices in Remote Locations
- Birkbeck-2011: Data Storage for Extreme Use Cases
- RefTest-2010: Has Mocking Gone Wrong?
- RBS-2009: Data Grids with Oracle Coherence
- Brunel-2008: The Architect's Two Hats
- Brunel-2007: Architecture and Design in Industry
- The Data Dichotomy (2016)
- Does In-Memory Really Make Sense? (2016)
- Elements of Scale: Composing and Scaling Data Platforms (2015)
- Upside Down Databases: Bridging the Operational and Analytic Worlds with Streams (2015)
- Log Structured Merge Trees (2015)
- Building a Career in Technology (2015)
- A World of Chinese Whispers (2014)
- Database Y (2013)
- The Big Data Conundrum (2012)
- Where does Big Data meet Big Database? (2012)
- A Story about George (2012)
- The Rebirth of the In-Memory Database (2011)
- Is the Traditional Database a Thing of the Past? (2009)
- Shared Nothing v.s. Shared Disk Architectures: An Independent View (2009)
- Component Software. Where is it going? (2005)
- Do Metrics Have a Place in Software Engineering Today? (2004)
Test Driven Development (all)
- Test Oriented Languages: Is it Time for a New Era? (2011)
- Beyond Stubs: Why We Need Interaction Testing (2010)
- Isolating Functional Units: Why We Need Stubs (2010)
- Are Mocks All They Are Cracked Up To Be? (2010)
Data Tech (all)
- Best of VLDB 2014 (2015)
- A Guide to building a Central, Consolidated Data Store for a Company (2014)
- An initial look at Actian’s ‘SQL in Hadoop’ (2014)
- The Best of VLDB 2012 (2012)
- Thinking in Graphs: Neo4J (2012)
- A Brief Summary of the NoSQL World (2012)
- ODC – A Distributed Datastore built at RBS (2012)
- Looking at Intel Xeon Phi (Kinghts Corner) (2012)
Team / Process / Interviewing (all)
- The Iffy Tractor (Can they code OO?) (2011)
- The Business Analyst Test (2011)
- Distributing Skills Across a Continental Divide (2011)
- Learning Practices for Distributed Teams (ICST) (2011)
- Interviewing: The Importance of Examining Applied Knowledge (2010)
- Mapping Personal Practices (2010)
- Four HPC Architecture Questions – With Answers (2009)
Slides from Strata Software Architecture: The Data Dichotomy – Rethinking data and services with streamsApr 5th, 2017
QCon 2017: The Power of the LogMar 8th, 2017
VIDEO (currently attendees only) HERE
This talk is about the beauty of sequential access and append only data structures. We’ll do this in the context of a little known paper entitled “Log Structured Merge Trees”. LSM describes a surprisingly counterintuitive approach to storing and accessing data in a sequential fashion. It came to prominence in Google’s Big Table paper and today, the use of Logs, LSM and append only data structures drive many of the world’s most influential storage systems: Cassandra, HBase, RocksDB, Kafka and more. Finally we’ll look at how the beauty of sequential access goes beyond database internals, right through to how applications communicate, share data and scale.
The Data DichotomyDec 14th, 2016
A post about services and data, published on the Confluent site.
This talk introduces Stateful Stream Processing and makes a case for SSP as a general approach to data computation in distributed environments.
Slides, alone, can be found here:
Slides from Codemesh & BigDataLdnNov 4th, 2016
Slides for JAX LondonOct 12th, 2016
Same title, but different content to the QCon one.
View full blogroll