The Curious Engineer

All blogs / Databases

Jesse Anderson

jesse-anderson.com

The Big Data and AI Expert for 50% of the Fortune 100. Sharing his years of experience and research to build high-performing data teams.

DatabasesDistributed systems / cloudEngineering management / careers

’10’15’20’25
Posts
243 from 12 Nov 2010 to 14 Nov 2025
Posting
now and then (1 in the last year)
Kind
personal blog
Feed
https://www.jesse-anderson.com/feed/

2025 6

  1. Gemini Batch API for Java
  2. Unapologetically Technical Episode 20 – Shane Murray
  3. Unapologetically Technical Episode 19 – Jacopo Tagliabue
  4. Unapologetically Technical Episode 18 – Adrian Woodhead
  5. Unapologetically Technical Episode 17 – Semih Salihoglu
  6. Unapologetically Technical Episode 16 – David Jayatillake

2024 13

  1. Unapologetically Technical Episode 15 – Frances Perry
  2. Unapologetically Technical Episode 14 – Cliff Crosland
  3. Data Teams Survey 2020-2024 Analysis
  4. Data Teams Survey 2024 Results
  5. Unapologetically Technical Episode 13 – Jeff Chou
  6. Unpacking the Latest Streaming Announcements: A Comprehensive Analysis
  7. Unapologetically Technical Episode 12 – AJ Hunyady
  8. Unapologetically Technical Episode 11 – Hubert Dulay
  9. Unapologetically Technical Episode 10 – Michael Drogalis
  10. Why Most Data Projects Fail & How to Avoid It at GOTO 2023
  11. Unapologetically Technical Episode 9 – Gunnar Morling
  12. Unapologetically Technical Episode 8 – Tom Scott
  13. The State of Data Engineering at Data Day Texas 2024

2023 14

  1. Unapologetically Technical Episode 7 – Stephane Derosiaux
  2. The Difference Between Learning and Doing
  3. Unapologetically Technical Episode 6 – Matteo Merli
  4. The Data Discovery Team
  5. Unapologetically Technical Episode 5 – Neil Avery
  6. Current 2023 Announcements
  7. GPT and LLMs from a Data Engineering Perspective
  8. Unapologetically Technical Episode 4 – Karthik Ramasamy
  9. Unapologetically Technical Episode 3 – Abhishek Chauhan
  10. Unapologetically Technical Episode 2 – Tomas Neubauer
  11. Data Teams Survey 2023 Follow-Up
  12. Unapologetically Technical Episode 1 – Roy Hasson
  13. Data Teams Survey 2023 Results
  14. Analysis of Confluent Buying Immerok

2022 10

  1. Brief History of Data Engineering
  2. Enabling The People, Enabling The Data with Kulani Likotsi
  3. See, Build, Test, Experiment: Using Data Science to Change the World with Erick Webbe
  4. Independent Anniversary
  5. The Art and Science of Data Storytelling with Brent Dykes
  6. Provoking Consumer-First Analytical Thinking with Drew Smith
  7. An In-Depth Data Mesh Discussion with Zhamak Dehghani
  8. The Reasons for Data Mesh on Pulsar
  9. The Soldiers, Rogues, and Mages of Data Teams
  10. KPIs Every Data Team Should Have

2021 6

  1. Ten Years On – The Million Monkeys Project
  2. Keeping Things Stupidly Simple With Pulsar and Kafka
  3. What Happens When Data Science Teams Add A Data Engineer
  4. Analysis of Confluent’s S1
  5. Why Data Science Teams Don’t Think They Need Data Engineering
  6. It’s Time to Change How We Manage Data Teams

2020 7

  1. Data Engineering Technology Tree
  2. Data Teams Survey Results
  3. What It Looks Like When a Team Is Missing
  4. Announcement: Data Teams Is Out!
  5. Kafka’s Got a Brand-New Poll
  6. Saving Money on Data Engineering in the Cloud
  7. Big Data and Analytics in the COVID-19 Era

2019 8

  1. Why I Recommend My Clients NOT Use KSQL and Kafka Streams
  2. Getting Your Programming Skills Ready for Data Engineering
  3. I Come Not To Bury Cloudera But To Praise It
  4. Reducing Operational Overhead with Pulsar Functions
  5. Reducing System Complexity with Event Sourcing
  6. Saving Money with Apache Pulsar Tiered Storage
  7. Q and A: Viewpoints on Open Source
  8. The Three Components of a Big Data Data Pipeline

2018 22

  1. Advice for Small Teams and Startups on Data Engineering
  2. Amazon and Open Source Business Models and You
  3. Creating a Data Engineering Culture
  4. Why You Can’t Do All of Your Data Engineering with SQL
  5. Thoughts on Cloudera Merging/Buying Hortonworks
  6. Creating Work Queues with Apache Kafka and Apache Pulsar
  7. Streamline Data Gathering
  8. InfiniteConf Keynote – Why Real-time is the Future
  9. What is a Data Pipeline?
  10. Professional Data Engineering Review – Sanjoy Roy
  11. Saying You Have Small Data Isn’t Belittling Your Use Case
  12. The Two Types of Data Engineering
  13. Why Real-time is the Future
  14. The Four Types of Technologies You Need for Real-time Big Data Systems
  15. What Are Batch and Real-time Big Data?
  16. Data engineers vs. data scientists
  17. Should You Even Do Big Data?
  18. Unit Testing Kafka Streams
  19. Are Your Programming Skills Ready for Big Data?
  20. The Veteran Skill on a Data Engineering Team
  21. How Much More Complicated Is Real-Time Big Data?
  22. Getting Into Big Data as a Consultant

2017 38

  1. Q and A: How do I improve my skills to become a Data Engineer?
  2. Why You Shouldn’t Write Your Own Distributed System
  3. What Is Big Data?
  4. You’re Probably Not a Distributed Systems Engineer
  5. On Cheating with Big Data
  6. When You Have the Wrong Team for Big Data
  7. Integration Testing for Kafka
  8. How Training is Delivered From the Beginning to the End
  9. This is Useless (Without Use Cases)
  10. Two Halves Don’t Make a Whole
  11. Apache Kafka and Amazon Kinesis
  12. Moving from OmniGraffle to SVG
  13. The Blame Game
  14. What It Looks Like From the Outside
  15. Medium Data
  16. Beam 2.0 Q and A
  17. The Difficulty of Transitioning to Data Pipelines
  18. Five Dysfunctions of a Data Engineering Team
  19. How to Evaluate an Open Source Product
  20. Kafka Topic Design Checklist
  21. The Many Meanings of Event-Driven Architecture: Kafka Edition
  22. Consumers and Creators of Technology
  23. There Are Several Hard Problems with Big Data
  24. Asking Better Questions
  25. The Learning Disconnect
  26. Doing Big Data ASAP
  27. What Happens When You Hire a Data Scientist Without a Data Engineer
  28. Personal Project Data Sources
  29. In-depth Interview
  30. Kafka REST and JQuery Helper
  31. How Are Programming and Distributed Systems Different?
  32. Announcement: Data Engineering Teams Book
  33. Is Kafka Only a Big Data Tool?
  34. What Do I Look for in Data Engineers?
  35. Q and A: Ingesting into Hadoop
  36. Hadoop MapReduce Dedupe Algorithm
  37. How much do companies lose before training?
  38. Maven Tips

2016 39

  1. Apache Beam Regex
  2. Beam’s Pico WordCount
  3. Are you attaining your goals?
  4. Unit Testing Kafka Consumers
  5. Unit Testing Kafka
  6. What will become of Big Data?
  7. Why should or shouldn’t you become a Data Engineer?
  8. Q and A: How can I tear out Informatica and MySQL and put in Big Data?
  9. Getting Stuck Crawling with Big Data
  10. Strata+Hadoop World and Trends
  11. Big Data Minority Scholarships
  12. Solving the First and Last Mile Problem With Kafka Part 2
  13. Solving the First and Last Mile Problem With Kafka Part 1
  14. How Programmers Should Start Viewing Training
  15. On complexity in big data
  16. Q and A: Is a Data Engineer the same thing as a BI or DBA?
  17. Crawl, Walk, Run with Big Data
  18. Q and A: Big Data strategy
  19. Is Big Data Cheap?
  20. Apache Kafka and Google Cloud Pub/Sub
  21. Kafka 0.10 Changes for Developers
  22. Question and Answers with the Apache Beam Team
  23. Ability Gap – Why We Need Data Engineers
  24. Big Data’s Required and Recommended Technical Skills
  25. My Big Data Journey
  26. The Case for Heron
  27. We Live, Eat, and Breathe This Stuff
  28. Spark and Java – Yes, They Work Together
  29. SSH With Google Cloud
  30. Announcement: Creating Big Data Solutions with Impala
  31. Unit Testing Spark with Java
  32. Three Top Themes From Strata+Hadoop World
  33. Kafka 0.9.0 Changes For Developers
  34. The ROI of the Right Training
  35. Hadoop Cheat Sheet
  36. Identifying Great Training Even If You Know Nothing About the Subject
  37. The Hidden Costs of the Wrong Training
  38. Is My Developer Team Ready for Big Data?
  39. Why You’re Not Getting a Data Engineering Job

2014 3

  1. The Cloudera Experience
  2. HBase With Playing Cards
  3. More NFL Posts

2013 17

  1. Hiring Your First Software Engineer
  2. Processing Big Data with MapReduce
  3. Cloudera QuickStart VM and Eclipse
  4. Guest Post on O’Reilly Programming Blog
  5. HBase REST Interface Part 3
  6. Propose, Prepare, Present Review
  7. Million Monkeys Interview on The Creators Project
  8. Interview in the RGJ
  9. HBase REST Interface Part 2
  10. HBase REST Interface
  11. Startup Communities Guest Post
  12. Optimizing MapReduce Jobs
  13. Social Recruiting The Best
  14. MapReduce and Graph Theory
  15. Mary Alice Berg (1936-2013)
  16. Big Data Congress
  17. NFL Play By Play Analysis

2012 8

  1. Articles In Pragmatic Magazine
  2. The Screencast Experience
  3. The Cloud and Amazon Web Services
  4. Kinder Gentler Apple
  5. Strata + Hadoop World 2012
  6. Hadoop The Definitive Guide 3rd Edition Review
  7. A Little Too Quiet
  8. EC2 Performance, Spot Instance ROI and EMR Scalability

2011 23

  1. Million Monkeys Visualization
  2. A Few Million Monkeys Randomly Recreate Every Work Of Shakespeare
  3. A Few Million Monkeys Randomly Recreate Shakespeare
  4. A Few More Million Amazonian Monkeys
  5. Pitching Agile
  6. Post Agile Checklist
  7. Oracle Database 9i, 10g, and 11g Programming Techniques and Solutions Review
  8. RIM Employee’s E-Mail To CEO
  9. Microsoft Kinect SDK
  10. A Million Amazonian Monkeys
  11. Googler Waves Goodbye
  12. Opinions on Interviews
  13. Apple’s Hardware and Software
  14. Life Without a Cellphone/Smartphone
  15. Changes at Google
  16. Hadoop Book Reviews
  17. Early Java Stories
  18. Bold Predictions
  19. Qt for Android
  20. Watson on Jeopardy
  21. Verizon+iPhone=Android Killer?
  22. Rooted Nook Color Review
  23. Kinect On Windows

2010 29

  1. Hacking the Kinect
  2. Pre-Interview (Job Interviews)
  3. The Technical Side of Personal Branding
  4. The (Sad) State of Java
  5. The Cost of Bad Code
  6. Keyboard Productivity
  7. Slightly-more-techinical-than-you Syndrome
  8. iPhone and Android Openness
  9. User Interface In Technology
  10. What Happened To Yahoo
  11. Code Review (For Interviews)
  12. Google Fu (For Interviews)
  13. Programming Job Interviews
  14. SSH Key Generation Instructions
  15. Courageous Followership for Programmers
  16. Give Chrome a Chance
  17. How to Find Crappy Programmers
  18. Is The iPad the Next Big Thing?
  19. Cross-Platform Is Not Limited to Java
  20. Initiative – The Key to Getting a Job
  21. Great Introduction to Source Control
  22. Analogy for Some Data Structures
  23. Rethinking IDE User Interface
  24. Maybe This Is Why Johnny Can’t Code
  25. Alternatives to MVC
  26. Why Johnny Can’t Code
  27. What Knowledge Gaps Do Self-Taught Programmers Generally Have?
  28. Morale the Tangible Intangible
  29. Friends Don’t Let Friends Copy and Paste