The Curious Engineer

All blogs / Python

Python⇒Speed

pythonspeed.com

Helping you deploy with confidence, ship higher quality code, and speed up your application.

PythonPerformance / optimizationRust

’18’19’20’21’22’23’24’25’26
Posts
100 from 4 Sep 2018 to 2 Aug 2026
Posting
monthly (6 in the last year)
Kind
personal blog
Feed
https://pythonspeed.com/atom.xml

2026 6

  1. Faster floating point math with Rust’s new API
  2. 8× faster binary search: from compiled code to mechanical sympathy
  3. Timesliced reservoir sampling: a new(?) algorithm for profilers
  4. Unit testing your code’s performance, part 2: Catching speed changes
  5. Speeding up NumPy with parallelism
  6. Unit testing your code’s performance, part 1: Big-O scaling

2025 5

  1. Testing the compiler optimizations your code relies on
  2. 330× faster: Four different ways to speed up your code
  3. Loading Pydantic models from JSON without running out of memory
  4. The surprising way to save memory with BytesIO
  5. Catching memory leaks with your test suite

2024 8

  1. Using portable SIMD in stable Rust
  2. Beyond multi-core parallelism: faster Mandelbrot with SIMD
  3. Let’s build and optimize a Rust extension for Python
  4. Let’s optimize! Running 15× faster with a situation-specific algorithm
  5. Not just NVIDIA: GPU programming that runs everywhere
  6. Profiling your Numba code
  7. Beware of misleading GPU vs CPU benchmarks
  8. NumPy 2 is coming: preventing breakage, updating your code

2023 17

  1. How many CPU cores can you actually use in parallel?
  2. Using Polars in a Pandas world
  3. Two kinds of threads pools, and why you need both
  4. Speeding up Cython with SIMD
  5. Speeding up your code when multiple cores aren’t an option
  6. The easiest way to speed up Python with Rust
  7. When NumPy is too slow
  8. Understanding CPUs can help speed up Numba and NumPy code
  9. Choosing a good file format for Pandas
  10. Goodbye to Flake8 and PyLint: faster linting with Ruff
  11. Polars for initial data analysis, Polars for production
  12. Speeding up text processing in Python (is hard)
  13. Python’s multiprocessing performance problem
  14. Don’t bother trying to estimate Pandas memory usage
  15. The problem with float32: you only get 16 million values
  16. Some reasons to avoid Cython
  17. Why Polars uses less memory than Pandas

2022 19

  1. Who controls parallelism? A disagreement that leads to slower code
  2. Good old-fashioned code optimization never goes out of style
  3. Early speed optimizations aren’t premature
  4. Find slow data processing tasks (before your customers do)
  5. Invasive procedures: Python affordances for performance measurement
  6. Finding performance problems: profiling or logging?
  7. The best way to find performance bottlenecks: observing production
  8. The limits of Python vectorization as a performance technique
  9. Finding performance bottlenecks in Celery tasks
  10. Pandas vectorization: faster code, slower code, bloated memory
  11. CPUs, cloud VMs, and noisy neighbors: the limits of parallelism
  12. Faster, more memory-efficient Python JSON parsing with msgspec
  13. When Python can’t thread: a deep-dive into the GIL’s impact
  14. Faster hardware is a bad first solution to slow software
  15. Processing large JSON files in Python without running out of memory
  16. Faster Python calculations with Numba: 2 lines of code, 13× speed-up
  17. The fastest way to read a CSV in Pandas
  18. How vectorization speeds up your Python code
  19. Memory location matters for performance

2021 12

  1. Creating a better flamegraph visualization
  2. Cython, Rust, and more: choosing a language for Python extensions
  3. NumPy views: saving memory, leaking memory, and subtle bugs
  4. Saving memory with Pandas 1.3’s new string dtype
  5. Measuring the memory usage of a Pandas DataFrame
  6. Measuring memory usage in Python: it’s tricky!
  7. Docker can slow down your code and distort your benchmarks
  8. The hidden performance overhead of Python C extensions
  9. Loading SQL data into Pandas without running out of memory
  10. Transgressive Programming: the magic of breaking abstractions
  11. All Pythons are slow, but some are faster than others
  12. Dying, fast and slow: out-of-memory crashes in Python

2020 16

  1. CI for performance: Reliable benchmarking in noisy environments
  2. Optimizing your code is not the same as parallelizing your code
  3. The mmap() copy-on-write trick: reducing memory usage of array copies
  4. Estimating and modeling memory requirements for data processing
  5. Debugging Python server memory leaks with the Fil profiler
  6. Too many objects: Reducing memory overhead from Python instances
  7. Massive memory overhead: Numbers in Python and how NumPy helps
  8. Clinging to memory: how Python function calls can increase memory use
  9. Debugging Python out-of-memory crashes with the Fil profiler
  10. Fil: a Python memory profiler for data scientists and scientists
  11. From chunking to parallelism: faster Pandas with Dask
  12. Fast subsets of large datasets with Pandas and SQLite
  13. Loading NumPy arrays from disk: mmap() vs. Zarr/HDF5
  14. Reducing Pandas memory usage #3: Reading in chunks
  15. The Parallelism Blues: when faster code is slower
  16. Copying data is wasteful, mutating data is dangerous

2019 16

  1. Reducing NumPy memory usage with lossless compression
  2. Reducing Pandas memory usage #2: lossy compression
  3. Reducing Pandas memory usage #1: lossless compression
  4. When your data doesn’t fit in memory: the basic techniques
  5. Beyond cProfile: Choosing the right tool for performance optimization
  6. Not just CPU: writing custom profilers for Python
  7. Where’s your bottleneck? CPU time vs wallclock time
  8. Speed is situational: two websites, two orders of magnitude
  9. Logging for scientific computing: debugging, performance, trust
  10. Choosing a faster JSON library for Python
  11. When C extensions crash: easier debugging for your Python application
  12. Realistic, easy, and fast enough: database tests with Docker
  13. When your CI is taking forever on AWS, it might be EBS
  14. Stuck with slow tests? Speed up your feedback loop
  15. Why Pylint is both useful and unusable, and how you can use it
  16. Fast tests for slow services: why you should use verified fakes

2018 1

  1. Why your multiprocessing Pool is stuck (it’s full of sharks!)