Myths Programmers Believe about CPU Caches
As a computer engineer who has spent half a decade working with caches at Intel and Sun, I’ve learnt a thing or two about cache-coherency. This was one of the hardest concepts to learn back in college – but once you’ve truly understood it, it gives you a great appreciation for system design principles. You might be wondering why you as a software developer should care about CPU cache-design. For o
On Disk IO, Part 1: Flavors of IO
If you like this story, check out my upcoming book on Database Internals! Series consist of 5 pieces: Flavors of IO: Page Cache, Standard IO, O_DIRECTMore Flavors of IO: mmap, fadvise, AIOLSM TreesAccess Patterns in LSM TreesB-Trees and RUM ConjectureNew series on Distributed System concepts in Databases can be found here. Knowing how IO works and understanding use-cases and trade-offs of algorith
Let's code a TCP/IP stack, 5: TCP Retransmission
At this point we have a TCP/IP stack that is able to communicate to other hosts in the Internet. The implementation so far has been fairly straight-forward, but missing a major feature: Reliability. Namely, our TCP does not guarantee the integrity of the data stream it presents to applications. Even establishing the connection can fail if the handshake packets are lost in transit. Introducing reli
How Postgres Makes Transactions Atomic
Atomicity (in the sense of “ACIDâ€) states that for a series of operations performed against a database, either every one of them commits together, or they’re all rolled back; no in between states are allowed. For code that needs to be resilient to the messiness of the real world, it’s a godsend. Instead of bugs that make it to production changing data and then leaving it permanently corrupt, those
Monitoring and Tuning the Linux Networking Stack: Sending Data | Packagecloud Blog
Monitoring and Tuning the Linux Networking Stack: Sending Data TL;DR This blog post explains how computers running the Linux kernel send packets, as well as how to monitor and tune each component of the networking stack as packets flow from user programs to network hardware. This post forms a pair with our previous post Monitoring and Tuning the Linux Networking Stack: Receiving Data. It is imposs
Modern garbage collection
You can find discussions on Hacker News and Reddit I’ve seen a bunch of articles lately which promote the Go language’s latest garbage collector in ways that trouble me. Some of these articles come from the Go project itself. They make claims that imply a radical breakthrough in GC technology has occurred. Here is the initial announcement of a new collector in August 2015: Go is building a garbage
Learning to Read x86 Assembly Language - Pat Shaughnessy
Writing assembly language is something best left for the experts. To write code that runs directly on your microprocessor you need to know how memory segmentation works, what the intended use of each register is, how codes executes in real and protected modes and much, much more. And, of course, modern compilers will usually produce faster, more optimized code than you ever could, without making a
Syscall Auditing at Scale - Slack Engineering
Electronic Circuit Board via Creativity103 licensed under Creative Commons. Cropped. If you are are an engineer whose organization uses Linux in production, I have two quick questions for you: 1) How many unique outbound TCP connections have your servers made in the past hour? 2) Which processes and users initiated each of those connections? If you can answer both of these questions, fantastic! Yo
Using CQRS with Event Sourcing or – what’s wrong with CRUD? – SoftMemes
Using CQRS with Event Sourcing or – what’s wrong with CRUD? November 12, 2016 By Kristian Freed 3 Comments CQRS, Event Sourcing, Java, Software Architecture, Software Trends The programming world is turning more and more functional. While pure functional programming languages like Haskell are yet to become mainstream, many of its tenets such as higher order functions and immutable data are – one i
Following a Select Statement Through Postgres Internals - Pat Shaughnessy
Captain Nemo takes Professor Aronnax on a tour of the engine room, a fascinating description of future technology from an 1870 perspective. This is the third of a series of four posts based on a presentation I did at the Barcelona Ruby Conference called “20,000 Leagues Under ActiveRecord.†(posts: one two four and video). Preparing for this presentation over the Summer, I decided to read through p
memory-configuration-scalability-blog-series
frankdenneman Follow Frank Denneman is the Chief Technologist for AI at VMware by Broadcom. He is an author of the vSphere host and clustering deep dive series, as well as a podcast host for the Unexplored Territory podcast. You can follow him on Twitter @frankdenneman Processor speed and core counts are important factors when designing a new server platform. However with virtualization platforms,
CPU Cache Essentials
This post came to my mind after watching the excellent presentation of Scott Meyers called "CPU Caches and Why You care". This post will try to summarize the ideas of the presentation so if you have some spare time you can just watch the presentation on video. To emphasize importance of cpu caches in our daily work we start with 2 examples: The first problem is a simple traversing of 2 dimensional
RAM is the new disk – and how to measure its performance – Part 2 – Tools - Tanel Poder Consulting
RAM is the new disk – and how to measure its performance – Part 2 – Tools [ part 1 | part 2 | part 3 ] In the previous article I explained that the main requirement for high-speed in-memory data scanning is column-oriented storage format for in-memory data. SIMD instruction processing is just icing on the cake. Let’s dig deeper. This is a long post, you’ve been warned. Test Environment I will cove
Reducing Memory Access Latency
Satoru Moriya <satoru.moriya.br@hitachi.com> Linux Technology Center Yokohama Research Lab. Hitachi, Ltd Copyright (c) 2013 Hitachi LTD., Systems Development Lab. All right reserved Reducing Memory Access Latency Copyright (c) 2013 Hitachi LTD., Systems Development Lab. All right reserved 2 Contents 1. Introduction 2. Issues 3. solution Copyright (c) 2013 Hitachi LTD., Systems Development Lab. All
Streaming Petabytes of Data in Realtime with Kinesis - NextRoll
AdRoll’s real-time data pipeline drives many systems crucial to our business. These systems inform the decisions made by our real-time bidding infrastructure in response to the tens of billions of daily requests we receive from ad exchanges. They also refresh our predictive models, guard against overspend, and deliver up to the minute campaign metrics through our dashboard. To function properly th
How to achieve low latency with 10Gbps Ethernet
Good morning! In a recent blog post we explained how to tweak a simple UDP application to maximize throughput. This time we are going to optimize our UDP application for latency. Fighting with latency is a great excuse to discuss modern features of multiqueue NICs. Some of the techniques covered here are also discussed in the scaling.txt kernel document. CC BY-SA 2.0 image by Xiaojun Deng Our expe
Cre8ive Thought
CQRS Ã la Greg Young I have had the pleasure of spending a 2 day course and many geek beers with Greg Young talking about Domain-Driven Design specifically focussed on the Command and Query Responsibility Segregation (CQRS) pattern. Greg has taken Domain-Driven Design from how Eric Evans describes it in his book and has adapted mostly the technical implementation of it. Command Query Separation (C
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
The Caveats of TCP_NODELAY
DevOps for Startups
Tuning Java Servers
{{#tags}}- {{label}}
{{/tags}}