![[syndicated profile]](https://www.dreamwidth.org/img/silk/identity/feed.png){kind=small}
justin_mason_feedApache Iceberg Internals Dive Deep On Performance
Good writeup on how Iceberg improves query performance across object storage, using predicate pushdown, manifest filtering, columnar vectorized reads, and file compaction.
Tags: iceberg internals file-formats data big-data object-stores storage formats columnar-storage predicate-pushdown performance
firedrake_feed2025 fantasy, third of what I believe is still expected to be a trilogy. Thara Celehar lost his ability to speak with the recently dead, but continues to serve his hierarchy and his god.
apnic_blog_feedtedu_inks_feedThis article, although in some parts very conversational, aims to demonstrate how it’s possible to build solid, valid, and efficient solutions without the need to use expensive and complex services. Moreover, this is the demonstration of how it’s possible to have your online presence without the need to put your data in the hands of third parties or without necessarily having to resort to complex stacks. Sometimes, less is more.
cks_techblog_feedWe are almost all of the way to the end of a multi-month process of upgrading our ZFS fileservers from Ubuntu 22.04 to 24.04 by also moving to more recent hardware. This involved migrating all of our pools and filesystems, involving terabytes of data. Our traditional way of doing this sort of migration (which we used, for example, when going from our OmniOS fileservers to our Linux fileservers was the good old reliable 'zfs send | zfs receive' approach of sending snapshots over. This sort of migration is fast, reliable, and straightforward. However, it has one drawback, which is that it preserves all of the old filesystem's history, including things like the possibility of panics and possibly other things.
We've been running ZFS for long enough that we had some ZFS filesystems that were still at ZFS filesystem version 4. In late 2023, we upgraded them all to ZFS filesystem version 5, and after that we got some infrequent kernel panics. We could never reproduce the kernel panics and they were very infrequent, but 'infrequent' is not the same as 'never' (the previous state of affairs), and it seemed likely that they were in some way related to upgrading our filesystem versions, which in turn was related to us having some number of very old filesystems. So in this migration, we deliberately decided to 'migrate' filesystems the hard way. Which is to say, rather than migrating the filesystems, we migrated the data with user level tools, moving it into pools and filesystems that were created from scratch on our new Ubuntu 24.04 fileservers (which led us to discover that default property values sometimes change in ways that we care about).
(The filesystems reused the same names as their old versions, because that keeps things easier for our people and for us.)
It's possible that this user level rewriting of all data has wound up laying things out in a better way (although all of this is on SSDs), and it's certainly insured that everything has modern metadata associated with it and so on. The 'fragmentation' value of the new pools on the new fileservers is certainly rather lower than the value for most old pools, although what that means is a bit complicated.
There's a bit of me that misses the deep history of our old filesystems, some of which dated back to our first generation Solaris ZFS fileservers. However, on the whole I'm happy that we're now using filesystems that don't have ancient historical relics and peculiarities that may not be well supported by OpenZFS's code any more (and which were only likely to get less tested and more obscure over time).
(Our pools were all (re)created from scratch as part of our migration from OmniOS to Linux, and anyway would have been remade from scratch again in this migration even if we moved the filesystems with 'zfs send'.)
apnic_blog_feedapnic_blog_feedhacker_news_daily_feedThe 10 highest-rated articles on Hacker News on April 22, 2025 which have not appeared on any previous Hacker News Daily are:
john_naughton_feed
Sadly, no. But when viewed from a distance it looked for a moment like a sinister predator surfacing in a placid lake!
“AI has by now succeeded in doing essentially everything that requires ‘thinking’ but has failed to do most of what people and animals do without thinking …”
Bach: Italian Concerto In F Major, BWV 971: I. Allegro | Rafał Blechacz
The Atomic Bomb Considered As Hungarian High School Science Fair Project
Very nice essay by Scott Alexander as he wanders down the rabbit-hole of the strange preponderance of Hungarian geniuses in the mid 20th century.
A group of Manhattan Project physicists created a tongue-in-cheek mythology where superintelligent Martian scouts landed in Budapest in the late 19th century and stayed for about a generation, after which they decided the planet was unsuitable for their needs and disappeared. The only clue to their existence were the children they had with local women.
The joke was that this explained why the Manhattan Project was led by a group of Hungarian supergeniuses, all born in Budapest between 1890 and 1920. These included Manhattan Project founder Leo Szilard, H-bomb creator Edward Teller, Nobel-Prize-winning quantum physicist Eugene Wigner, and legendary polymath John von Neumann, namesake of the List Of Things Named After John Von Neumann.
The coincidences actually pile up beyond this. Von Neumann, Wigner, and possibly Teller all went to the same central Budapest high school at about the same time, leading a friend to joke about the atomic bomb being basically a Hungarian high school science fair project.
But maybe we shouldn’t be joking about this so much. Suppose we learned that Beethoven, Mozart, and Bach all had the same childhood piano tutor. It sounds less like “ha ha, what a funny coincidence” and more like “wait, who was this guy, and how quickly can we make everyone else start doing what he did?”
In this case, the guy was Laszlo Ratz, legendary Budapest high school math teacher…
With Alexander, often the journey not the arrival matters most. And so it is here. But you need to read all the way to find that out. Personally, I couldn’t put it down.
And if you’re still intrigued, there’s also ”The Martian Gene” by David Friedman to keep you from doing real work!
The Thinking Machine: Jensen Huang, Nvidia and the World’s Most Coveted microchip
My Observer review of the book in last Sunday’s edition of the paper:
This is the latest confirmation that the “great man” theory of history continues to thrive in Silicon Valley. As such, it joins a genre that includes Walter Isaacson’s twin tomes on Steve Jobs and Elon Musk, Brad Stone’s book on Jeff Bezos, Michael Becraft’s on Bill Gates, Max Chafkin’s on Peter Thiel and Michael Lewis’s on Sam Bankman-Fried. Notable characteristics of the genre include a tendency towards founder worship, discreet hagiography and a Whiggish interpretation of the life under examination.
The great man under Witt’s microscope is the co-founder and chief executive of Nvidia, a chip design company that went from being a small but plucky purveyor of graphics processing units (GPUs) for computer gaming to its current position as the third most valuable company in the world.
Two things drove this astonishing transition. One was Jensen Huang’s intuitive appreciation that Moore’s law – the observation that computing power doubles every two years – was not going to apply for ever, and that a radically different kind of computing architecture would be needed. The other was his decision to bet the future of Nvidia on that proposition and turn the company on a dime, much as Bill Gates had done with Microsoft in the 1990s when he had realised the significance of the internet…
And just in case you are in Trumpland and thinking to doing some research on misinformation, this note from the National Science Foundation might give you pause. The regime is not going to fund research on such timely and relevant subjects. This is real authoritarianism at work, with the cooperation of hitherto independent institutions.
This Blog is also available as an email three days a week. If you think that might suit you better, why not subscribe? One email on Mondays, Wednesdays and Fridays delivered to your inbox at 5am UK time. It’s free, and you can always unsubscribe if you conclude your inbox is full enough already!
justin_mason_feedE-ink IBM XT clone "with solar power, ultra low power consumption, and ultra long battery life: in power saving mode it can run between 200 hours on the low side and 500 hours or in some cases even much longer of constant interactive use, not standby." -- this is an absolutely crazy gadget. I never thought I'd feel nostalgic for MS-DOS, but here we are
Tags: pc e-ink solar retrocomputing emulation hardware gadgets self-builds ibm-xt solarpunk
matklad_feedWhat’s in common between these two blog posts (pardon me, my fellow crustaceans of lobster variety)?
They both have horizontal scroll on mobile! Horizontal scroll is very distracting – when you scroll vertically with your thumb, you necessarily scroll horizontally as well. Any amount of horizontal scroll clips the left edge of the screen, chopping of begging of every sentence!
This is a hard problem to fix, or at least I don’t know a reliable simple solution. Still, I think this site should not have overflows, so let me share some things I’ve learned about the width of things!
In general, you want your main text to be pretty narrow, about 50-60 characters. This creates a problem for coding blogs — code, unlike prose, wants to be a bit wider. While its a good idea to keep most of the examples narrow, an occasional line still wants to be longer than 60. My suggested solution here is to accept this, and allow code samples to be wider than the main text. This is not super trivial to achieve with CSS, but I borrowed a solution from https://blog.xoria.org.
My HTML structure looks roughly like this, body > main > article > section? > p:
<body>
<main>
<article>
<h1>
Against Horizontal Scroll <time class="meta" datetime="2025-04-22">Apr 22, 2025</time>
</h1>
<p>
What’s in common between these two blog posts (pardon me, my fellow
crustaceans of lobster variety)?
</p>
<figure class="code-block">
<pre><code>A bunch of code here.</code></pre>
</figure>
</article>
</main>
</body>And I use the following CSS for narrow main text width and moderately wide code:
/* Set the body width to how wide you want the code to be */
body {
max-width: 80ch;
padding: 2ch;
margin-left: auto;
margin-right: auto;
}
/* Then, limit every "paragraph" of article to be narrower.
* Paragraph is just any descendant of the main article tag.
*/
article > *,
article > section > *
{ max-width: 55ch; }
/* Finally, opt-out of narrow width for figures with code */
article > section,
article > figure,
article > section > figure
{ max-width: 80ch; }Still, it is hard to enforce that all code samples fit within the limit, and there are narrow screens where they can’t physically fit.
To solve this, you need to make sure that, when a horizontal scroll-bar appears due to a wide code block, it only appears on the code block, and not on the entire website:
figure.code-block > pre > code {
overflow-x: auto;
}Code blocks are the primary cause of horizontal scroll, so it’s great that this is fixed now! Two more left!
The browser can reflow your article to make it narrower by breaking text over multiple lines. This requires that your words are not too wide. This is usually not a problem, unless you are a German, but there are certain technical “words” that are, in fact, wide. For example, I like to use bare URLs in my writing, and, because urls typically do not contain spaces, they can’t be broken over multiple lines and cause scrollbars. For such cases, you need to inform the browser that it is okay to break the thing anywhere:
p { hyphens: auto; }
a.url { word-break: break-all; }The final issue that hit me relates to flexbox. Specifically, its min-width semantics. If you
don’t specify min-width of a flex item explicitly, it gets defaulted to min-width:
auto. This means that the minimum width that the item can occupy is determined by
not breaking paragraphs in lines! This way, the computed minimal width of the element ends up
being much wider than what you would naturally expect. To fix it, manually override it:
aside.admn > div { flex: 1; min-width: 0; }That’s all for today. To underscore my point, and in solidarity with the authors of the two blogs
that started this, let me include an url without word-break to make sure this article itself has
the problem it talks about:
domainincite_feedA Chinese clothing company has asked ICANN to delete its dot-brand gTLD, despite the fact that it is being used for web sites and email. Redstone Haute Couture wants rid of .redstone, which has been in active use for almost a decade. My database shows that it has about a dozen names, most registered in […]
The post Dot-brand actually being used to get deleted first appeared on Domain Incite.
talks_cl_cam_feedAbstract not available
roads_org_uk_feedThe tunnel London loves to hate is finally here. On Monday 7 April the new A1026 Silvertown Tunnel opened for business. One question remains: what’s it for?
The new tunnel crosses the Thames between North Greenwich and Silvertown - right beside the O2 Dome - linking the existing A102 Blackwall Tunnel Southern Approach with the Lower Lea Crossing, and in doing so providing a new way to cross the river near the chronically congested Blackwall Tunnels.
To say this is a rare event in London would be an understatement. For decades the city has been consistently working to reduce car use. Until the Silvertown Tunnel opened, the body that built it - Transport for London, which oversees all public transport in the capital along with its major road network - had built just one road in its 25 year history, a mile-long 30mph single carriageway bypass called the Coulsdon Relief Road. The construction of a £2bn dual carriageway tunnel is decidedly out of character.
Besides, any new road in London will be controversial. Londoners have fought roadbuilding tooth and nail since the late 1960s, but even in that context, the Silvertown Tunnel is remarkable for the universal opposition it faced. Usually some groups dislike a new infrastructure project while others are in favour, but somehow this is a road that absolutely nobody seems to like. So why did TfL do it?
Most of the debate - especially the attention-grabbing hot takes being thrown around on social media - feeds off people’s own ideas about what it’s for, perhaps because TfL and the Mayor haven’t gone out of their way to explain it. Publicity refers to faster cross-river journeys and better public transport links, neither of which offers enough detail to explain why it was built.
We’ll uncover the rationale for the Silvertown Tunnel in a moment. But before we do, perhaps we should explore some of the things that are making people so angry.
The Mayor of London, Sadiq Khan, has made air quality a key plank of his work, to the extent that he wrote and published a whole book that frames it as a personal mission. It’s central to his policy platform.
No surprise, then, that people are asking why he has simultaneously overseen construction of a new road tunnel, which will increase road space for private motor traffic.
Isn’t that contrary to his stance on air quality? Doesn’t it fly in the face of his work on London’s Ultra Low Emission Zone? This is, after all, a project he inherited: when he was elected Mayor in 2016, work had yet to start. He could have binned it if he wanted, but he didn’t - in fact he has invested significant political capital in pushing it through in the face of opposition. The loudest and most persistent anti-tunnel campaigns all focussed on its potential to worsen air quality around the tunnel and its approaches.
Public transport and active travel (meaning walking and cycling) are central to TfL’s work. The Silvertown Tunnel does accommodate public transport; two of its four lanes are reserved for buses and heavy goods vehicles, a feature settled early in the design process, and new bus routes have been created that use it. But there is no provision for walking or cycling.
Well, that’s not entirely true. There is the Silvertown Cycle Shuttle, which takes cyclists across the Thames for free. It’s an electric bus route, running about every quarter of an hour, between stops on either side of the river.
Cycle campaigners are not impressed, and understandably so. There are already ways to cross the river by loading your bike onto some other form of transport. The Shuttle doesn’t greatly improve their options; indeed, if you wish to use it for a serious journey, the time taken to reach its terminus, wait for the next bus, ride across and then get moving again represent a significant amount of inconvenience. It’s fair to say that it will be unsurprising if the Shuttle fails to draw crowds.
A new tunnel was a chance to provide a route for cyclists to simply ride across - either by building a foot and cycle tunnel, or by closing the ancient and twisty northbound bore of the Blackwall Tunnel to motor traffic (something it’s never been well suited to carry) and repurposing it for active travel. The lack of provision, in other words, is seen as proof that the Silvertown Tunnel is a regressive project that’s all about private motoring.
The new tunnel arrives on the north bank of the Thames at a messy signalised roundabout. From there, all its onward connections are cumbersome. The main approach is the Lower Lea Crossing, a dual carriageway bridge over the mouth of the River Lea that had plenty of spare capacity, but journeys between the tunnel and the A13 or the Docklands all have to negotiate a busy roundabout and multiple sets of traffic lights.
If the new tunnel is to be used to its maximum effect, it needs to be quicker and more convenient than Blackwall for those journeys. But these design choices don’t seem to achieve that, presenting a risk that traffic will continue to prefer Blackwall. Equally, if Silvertown does attract intensive use, critics fear that its awkward approaches could be easily overwhelmed.
If you’re going to all the trouble of building a new mile-long tunnel under the Thames, the argument goes, it needs to be easier to reach than this.
East London only has so many river crossings for road traffic. There’s Tower Bridge, which is a historical monument with a weight limit, and the Rotherhithe Tunnel, which was designed for horse traffic and is narrow enough to be mistaken for one of those little pipes that hamsters run through. Then there’s Blackwall, and after that it’s eleven miles as the crow flies to Dartford where M25 traffic finds a way across.
In that eleven mile span there is only the Woolwich Free Ferry, which is fine as far as it goes but has extremely limited capacity.
Why, then, is this new river crossing right next door to Blackwall, leaving that eleven mile gap untouched? Wouldn’t it be better to have provided an option to cross in a new location, bringing those disparate districts on the north and south banks closer together?
Back in 2016 Transport for London actually consulted on three river crossings and at one stage thought it might build them all: they were Silvertown, Gallion’s Reach and Belvedere. But only Silvertown got any further. If you could only build one, why build it where a crossing already exists?
Those who are content to see more road capacity across the Thames are unhappy that the new road is 50% bus lane. Even though TfL has instituted new routes to make use of the tunnel, there are still only so many buses that come this way. Is it worth reserving so much valuable road space for them?
In truth these are not just bus lanes, they are open to heavy goods vehicles too, so it won’t only be buses using them. But even so, if you use the tunnel in these first days, there is plainly one lane in use and another seemingly empty, reserved for just the occasional bus.
If you are spending all this money to build a twin-bore tunnel, if you are going to all this trouble - don’t you want to make the available space work as hard as it possibly can?
This one is really simple. Blackwall used to be free. Now, to pay for the new tunnel, both are subject to a toll.
Motorists object to being charged to use an existing tunnel that has been free since 1897, and that is surely understandable. But there is also a question about whether the toll will push traffic away to other less suitable crossings - the Rotherhithe tunnel, perhaps, or Dartford where the toll is cheaper. None of the alternatives have capacity to spare.
The Silvertown Tunnel comes with a price tag of £2bn. Public transport campaigners balk at the money that has been spent increasing capacity for motor traffic. Those happy with the new road space find it hard to believe so much money has been spent to provide so little new road and such limited connections on the north bank.
Active travel campaigners, meanwhile, can only look on and despair at how this sum for a single road project compares to the budget for cycling and walking improvements across the whole of London.
The list above is a mix of complaints from people with all sorts of differing ideas about transport in London. It’s probably worth acknowledging, before we even start looking at the reasons TfL built the Silvertown Tunnel, that some of them are contradictory.
You are, for example, either upset with the cost of the tunnel or with the tolls, but it’s hard to object to both. The tolls were introduced to pay for it, so while £2bn is a huge amount of money for TfL to spend on a new road, it was paid for by a loan that drivers will repay in years to come. It was never money that could have been spent instead on tube upgrades or cycle facilities, nor would the tunnel have ever been built if the tolls weren’t there.
Equally, those who object to the bus and HGV lanes are unlikely to be the same people who find the Cycle Shuttle inadequate.
So why was it so important for an organisation that actively discourages travel by road, and a Mayor who is determined to tackle air pollution from motor vehicles, to face down opposition from every quarter and build a new road tunnel right next to an existing one?
The question can only be answered by finding the reasons it was built. For those, we will turn to the User Charging Assessment Framework, a document published late last year. It lists the project objectives and then summarises the forecasts that have been made for each one. (Its forecasts should be the most accurate available, because they take account of the exact toll charges which were only finalised recently.)
The following are the things that the Silvertown Tunnel was built to do. Right now, we only have TfL’s forecasts to measure them against, and it remains to be seen how the opening of the new tunnel will truly affect travel patterns, and indeed life, in East and South East London. But the forecasts do provide an answer to some of the complaints and concerns that are directed at it.
These three objectives are all about journey times. The Blackwall Tunnels have been the cause of chronic, unbearable congestion for many years, and fixing that problem is not simply about enabling more people to have nicer journeys by car. The lack of options for crossing the river in East London mean that Blackwall was one of the only ways for goods, services, buses and other essential traffic to get across the Thames. The fact that it was so unbelievably oversubscribed had become a limiting factor in the performance of the road network of the whole of East and South East London.
It’s not just about capacity, either. It’s about resilience. Blackwall’s northbound bore, opened in 1897, is tiny and has a number of sharp bends. That makes it prone to accidents, and accidents mean delays at best and tunnel fires at worst. When a vehicle arrives that is too high or wide to use it, the whole approach road has to be held while it is diverted off. The southbound bore is better but still has some restrictions. As a result, Blackwall was not just congested but also deeply unreliable; indeed TfL have stated before that the tunnels only operated “normally”, without an incident that required intervention or even temporary closure, on one day in every two weeks.
So is the tunnel in the wrong place? Well, it’s in the right place to solve the problems that TfL were trying to solve at the Blackwall Tunnel. The lack of river crossings elsewhere is a separate problem, and Silvertown was never meant to fix that. You could argue that if only one crossing could be built, it should have been elsewhere, but to do that, you would need to accept that Blackwall’s problems, with all their social and economic knock-on effects, would not have been addressed.
And that problem will be addressed: we can be pretty certain it will fix the lack of resilience at Blackwall. (Cherish this clear-cut answer, there’s precious few of them in this game.) It provides a second physical route each way, so if one tunnel is closed the other remains open, massively reducing the impact when something goes wrong.
TfL estimate that morning rush hour journeys in “normal” conditions will be at least 15 minutes faster, and time savings will be even greater than that when the tendency for Blackwall to be closed or disrupted is taken into account. Journey times will become predictable.
Traffic levels overall are expected to rise slightly - a 2% increase is forecast for the first year of operation, since the route will become more attractive. Research over the last couple of years, including consultation with motorists who use the tunnels, indicates that this time saving will be worth more than the cost of the toll, so most will calculate that it’s more worthwhile to pay it and drive through than to abandon their journey or divert to another route. The effect of the toll is mainly in capping the amount of additional traffic that would otherwise be generated.
The net result is that there is expected to be roughly the same amount of traffic as before, but now with 50% more capacity to cross the river and fewer disruptive incidents to hold it up.
If traffic levels do remain roughly static, then in normal use the Lower Lea Crossing and its roundabouts and traffic lights will only be handling about a third of the traffic volumes currently passing through Blackwall. In any case, Silvertown has just one general traffic lane in each direction, so the total traffic load it can feed into those approaches is limited and is much lower than Blackwall itself. When Blackwall has to be closed, however, and Silvertown is the only available route, you can expect long delays, since its one lane will be better than nothing but no substitute for three.
Because few people are forecast to be deterred by the tolls, only marginal increases in traffic are expected at other river crossings (1% each at Rotherhithe and Dartford). Meanwhile, significant improvements are expected on approach roads thanks to reduced congestion - particularly on notorious rat runs north of the Thames like Chrisp Street, which passes through residential and shopping districts and often carried significant amounts of traffic trying to avoid the queues for Blackwall.
Finally, it’s clear that active travel simply wasn’t an objective for the scheme. Whether it should have been is open to debate, but since resilience is central to the project, and that’s delivered by having two tunnels in each direction, there is no hope of closing one of the Blackwall bores for active travel use. Including a cycle crossing would therefore have required an extra tunnel bore - a significant extra cost that might have made the project untenable. That doesn’t excuse the absence of active travel provision but it perhaps explains why TfL came up with the Cycle Shuttle.
Blackwall offered no room for growth - indeed it had become a critical limiting factor on the ability of East and South East London to grow. Even a city that actively discourages motor traffic must accommodate a certain amount of it; as the city grows that amount will go up, at least a bit. In most of London the existing network will absorb that increase one way or another, but Blackwall was long past that point. It is a rare place where TfL concluded that London really did need at least some extra road space to support its spiralling population.
The reduction in congestion is predicted to total 9,800 hours a day for drivers and 2,800 hours a day for public transport users - time that can now be used productively instead of idling in traffic.
Perhaps more importantly, the tunnel will save 5,800 hours a day for commercial vehicles, principally by enabling direct journeys across the river. Until now, companies have had a logistical nightmare arranging their supply chains because East and South East London were effectively two separate worlds; without paying the Congestion Charge the nearest crossings open to HGVs were Dartford, Vauxhall or the Woolwich Ferry. That made business inefficient and introduced an unbelievable penalty in extra time and distance, not to mention the lorries thundering through other parts of London that shouldn’t need to be there.
Right now the number of lorries using the Silvertown Tunnel is relatively small, because supply chains have yet to adapt to it, but the benefit it will eventually provide could be considerable and widespread.
In this light the decision to reserve half the tunnel’s capacity for bus and HGV lanes can be understood. If TfL’s forecast is right, the tunnel’s two general traffic lanes will supply enough new capacity to cater for the current level of demand, which is capped by the toll. The bus and HGV lanes prioritise the traffic that TfL considers to be most critical - public transport and goods - and protects it from the effects of congestion. In turn London’s other river crossings and street network are protected from HGVs diverting long distances to cross the river elsewhere, as they do now, and supply chains are given the greatest possible guarantee of reliable journey times.
This amounts to making North Greenwich a nicer place to be, since it is currently being redeveloped into a high-rise residential and entertainment district. The elimination of the all-day tailback through the middle of it is expected to make it noticeably more appealing and more accessible.
Ah yes, this is the big one. Under this heading you will find air quality.
While there is a forecast 2% rise in traffic levels, those vehicles are now expected to be moving, not sitting idle for long periods, and consequently air quality is actually forecast to improve.
In some places.
A bit.
The Blackwall Tunnel Northern and Southern Approaches will see the greatest benefit - naturally, since they have been all-day car parks until now - with a decrease in nitrogen dioxide concentrations between 2 and 3 micrograms per cubic metre.
Improvement, by smaller margins, is also expected on the A13 and around residential streets historically used as rat runs. Slight increases are forecast at the Tidal Basin Roundabout, where the tunnel emerges on the north bank, and at some odd spots further afield.
In general, TfL’s assessment on air quality is mostly listed as “no impact”, with some areas showing “slight positive change”. If their forecasts are broadly right, the tunnel will not be the air quality disaster that some have feared.
That’s unlikely to appease anyone campaigning against the tunnel on air quality grounds, since the Blackwall Tunnels and their approaches were already an air quality nightmare. No impact is not an improvement, it’s the status quo, and the status quo is not good. If it hadn’t been built, other interventions could have been made that might have actually improved air quality.
However, the status quo is at least no worse than before. Since the new tunnel is now a fait accompli, the promise of fewer idling vehicles and less traffic in surrounding neighbourhoods would at least be welcome, and the Mayor can claim with justification that the tunnel is not cancelling out his efforts on air quality. But while no disaster is foreseen, there is also no great cause for joy on this front.
Also under this heading is the impact on lower income communities. TfL’s new bus routes are designed to serve people who would otherwise feel little benefit - 60% of those within reach of bus stops on the new routes have no access to a car. The Cycle Shuttle is free.
Care has also been taken to mitigate the effect of the tolls on those who can least afford the increased cost. There are significant discounts for local residents in receipt of benefits, and the toll is waived for Blue Badge holders, coaches, taxis, private hire vehicles in service and others. And for the first year at least there is a discount for small businesses and charities in neighbouring Boroughs.
We can skip this one. TfL had a word with the Boroughs, the City and National Highways, and they all think it’s OK. Let’s move on.
It’s too early to say whether the Silvertown Tunnel will work out as TfL and the Mayor hope. People, and traffic patterns, are too unpredictable for that, and we’ve yet to see how it operates outside of the Easter holidays, let alone when travel patterns settle into new shapes over the coming months and years.
But what might be clearer now is the job it’s supposed to do. It’s not about capacity, it’s about reliability - finally conquering the drag that the heavily restricted and highly disruptive Blackwall Tunnels have had on East and South East London for decades.
Besides, Transport for London is not a road builder and it is not reckless. You may doubt the need for two more traffic lanes across the Thames, but consider the scarcely believable fact that TfL are the ones who built them.
This one road project - conceived in the 1990s, but descended from an earlier idea dating back to the 1960s to provide a third bore or a bridge at Blackwall - survived the total attrition of all other road projects in London. It survived TfL’s financial crisis and the ending of central Government grants for London’s road network, and received no Treasury funding whatsoever. It survived the culling of its stable mates, Gallion’s Reach and Belvedere, both considered unaffordable. It survived a change in leadership, from a bullish Conservative Mayor with a well-known penchant for bridges and tunnels to a Labour Mayor whose stated goal is to clean up London’s air.
The Silvertown Tunnel is, in other words, a unicorn. It shouldn’t be possible for it to exist. The fact that it does - that in a city where streets are routinely narrowed in favour of wider pavements and cycle facilities, where even maintaining the current level of capacity for motor traffic is unthinkable, let alone increasing it - that fact should tell you just how certain TfL must be that here, in this one place, London really, really did need a new road.
Is this a defence of the Silvertown Tunnel in the face of all its critics? No, or at least not entirely. But there has been a lot of noise and a lot of hasty judgement of a project that will take years to fully bed in and prove itself. So this is an attempt to provide a more thoughtful perspective, one that takes into account the reasons the tunnel was built and the things it is actually expected to do.
Was TfL right that this issue, above all others, was the one that had to be solved? And will the Silvertown Tunnel fix it? We don’t know. But it’s here now, like it or not, so let’s give it time, and see what this unicorn does.
smbc_comics_feed
Hovertext:
The thing I miss most about the 90s is the ability to time travel.
nwhyte_atom_feedI like Simon Guerrier; up to now I have generally liked his writing; I love Peter Purves both as Stephen and playing the Doctor; in the week when we got the sad news of Jean Marsh’s death, it was lovely to hear her resuming the role of Sara Kingdom; and the story of the Doctor’s first encounter with Sontarans – proper bloodthirsty Lynx and Styre type Sontarans – is well structured and well told.
But I am afraid I don’t like torture scenes, and although of course it’s perfectly consistent with Styre in The Sontaran Experiment, I didn’t like that much either. So it’s a rare thumbs down for me for this particular combination of creators.
schneier_no_tracking_feedAndroid phones will soon reboot themselves after sitting idle for three days. iPhones have had this feature for a while; it’s nice to see Google add it to their phones.
drmaciver_feedAlgebra and insight
I was reading up on Markov Chains today, as one does, partially reminding myself about them and partly filling in some gaps in my knowledge.
One bit I was reading about was reversible distributions. If you’ve got some transition matrix \(P_{ij}\), a reversible distribution for it is a probability distribution \(q\) over the states such that \(q_i P_{ij} = q_j P_{ji}\). If you also lack any intuition about what that means, me too at the point at which it’s introduced. The broad intuition is that a Markov chain in a reversible distribution can be “run backwards in time” and you can’t tell the difference, but the main significance is that it’s an easy property to verify that has strong consequences.
Specifically, it’s a theorem that every reversible distribution is stationary. That is, if \(q\) is reversible then \(q_i = \sum\limits_j P_{ji} q_j\) for all \(i\) - the distribution is not changed by running the Markov chain forwards.
The proof of this presented in the text I was reading is as follows:
We can write \(q_i = q_i \sum\limits_j P_{ij}\) (because it’s a standard property of transition matrices that this sum is 1), so \(q_i = \sum\limits_j q_i P_{ij} = \sum\limits_j q_j P_{ji}\), so \(q\) is stationary.
I can follow this proof of course. It’s very simple algebra. But there’s something about it that feels like a magic trick. I don’t feel like I’ve got any actual insight into the nature of reversible distributions or have any sense of why they must be stationary from this. I’m sure there are plenty of mathematicians who could, and I’m not even sure that I need that insight, but it bugged me.
Thinking about it a bit more lead me to come up with the following alternative proof<input ... >Which I suspect I have cribbed in some sort of half-remembered manner from Geoffrey Grimmett’s “Probability on Graphs”, but if so it was useful to recreate it from memory anyway.
One way to think about distributions on markov chains is that you’ve got a certain amount of “stuff” (probability mass) floating around, and at each step that you run the Markov chain, this probability mass flows around the graph.
When you’ve got a probability distribution \(q\), all of \(q_i\) flows out of node \(i\), and \(\sum\limits_j q_j P_{ji}\) flows into it. That is, for each node \(j\), the mass in it flows out to every other node, and the fraction of it that goes to \(i\) is \(P_{ji}\).
A distribution is stationary if for all \(i\), \(q_i = \sum\limits_j P_{ji} q_j\), but equally we can write this as \(q_i - \sum\limits_j P_{ji} q_j = 0\). The net flow of mass out of \(i\) is 0.
We can write this flow of mass out of \(q_i\) as \(q_i = \sum\limits_j q_i P_{ij}\) - the sum of the mass flowing to each \(j\). So the net flow of mass out of \(i\) is \(\sum\limits_j P_{ij} q_i - P_{ji} q_j\). That is, there is a net flow along each edge, let’s say \(f_{ij} = P_{ij} q_i - P_{ji} q_j\). The probability of \(i\) remains unchanged if \(\sum\limits_j f_{ij} = 0\).
However, the reversability condition is precisely that all of the \(f_{ij} = 0\). That is, there is no net flow of mass along each edge. As a result, a reversible distribution is necessarily stationary, because a much stronger condition applies: Not only is the total net flow across the edges from \(i\) equal to 0, it’s 0 along each edge.
This proof is in some sense much worse. It’s longer and involves more algebra rather than less. But after reading this proof I felt like I understood why reversibility obviously implied a distribution was stationary, and also why it was a much stronger condition than being stationary.
I think mathematics - and many other things - benefits hugely from this sort of chewing on a problem and seeing it from different angles until it actually makes sense. If you just know that a result is true, you often find yourself unable to really use it. If you understand at an intuitive level why it’s true, it will stick with you and you can really see how to apply it, including in cases where it doesn’t quite hold but something similar does. Proof without developing insight alongside it is certainly better than no proof, but it’s in some sense fundamentally fragile and untrustworthy, and it’s often better to mull it over until you find the right way to break up the problem to actually allow it to make sense at an emotional level.
scalziwhatever_feed
Any author can tell you that events in their own life can have an impact on their fiction. As we learn in Heather Tracy’s Big Idea for Only a Chapter, sometimes those events have a bigger impact than we might have expected.
HEATHER TRACY:
When I began writing what would become Only a Chapter back in 2015, the working title I had then was “Faceless Man.” I knew I wasn’t going to call the book that, but I couldn’t come up with anything better. I still have several drafts of the original version saved with that name on my computer.
The big idea for the original version of the book came from dreams I had in high school through college of a faceless man who would do huge romantic things like fly me on a private jet to New York City to see Phantom of the Opera on Broadway with the original cast, then he proposed. The dreams were always very vivid, and I could always tell the man was wearing a tuxedo, but I could never see his face. Sometime after dating my now-husband for a while, I realized that when he and I originally met at my senior prom, he was wearing a tux. In different ways, a lot of the things in my dreams did happen, but much less sensationally. For instance, before he proposed, he took me to see a local production of A Chorus Line.
In “Faceless Man,” Clare had these dreams, they pointed her to this dream guy, and that was about it. The story was fun, but pretty flat. There wasn’t enough heart. There wasn’t enough tension. I put the book to the side for almost nine years.
Then, after completing breast cancer treatment in early 2023, big idea number two hit me (seriously, I can never have just one big idea for these things): What would happen if Clare had breast cancer, but also, what would happen if she didn’t? What if the story had two timelines with the ways her life could go if that dreaded phone call went two different ways? I had obviously been contemplating this scenario in my own life and thought it would be therapeutic to work it out through my fiction.
The final version of the book still has the faceless-person dreams, but this time, they’re different depending on the timeline. Clare’s bisexual, and in one timeline the dreams start pointing her toward a male, and in the other a female. In the timeline where she has breast cancer, the cancer diagnosis and story are my own, though fictionalized slightly to work within the confines of the narrative.
Oh, and the title? When I announced on social media that I had breast cancer back in 2022, I said on social media that “Cancer is only going to be a chapter in my life, and not the whole story.” Thus, Only a Chapter was born.
Only a Chapter: Amazon|Space Wizard
scalziwhatever_feed
Ever have that dream where your teeth fall out? Well, it’s not a dream in my case; last night, while chewing, one of my crowns tried to escape. Fortunately I realized what was happening before I bit down, and therefore saved the thing for the appointment my accommodating dentist arranged for me this morning.
The good news is the crown is now safely back in my head; the less great news is now this formerly-permanent crown is a temporary, and I have to go back in a couple of weeks to get a new permanent crown. Dentistry is confusing, y’all.
Anyway, that’s been my last 15 hours. How are you?
— JS
dotaturls_feedjustin_mason_feednotes on using an LLM for personal email search
Nelson Minar asked Mastodon about using an LLM for email search over "20+ years of email archives":
"Main use would be a query for specific things, "what did I say to this friend 10 years ago about music?" But also just for general knowledge. I think it'd mostly work as free text but there's a little email-specific structure it'd be nice to capture."
The thread has some good suggestions, notably Mark Fletcher's RAG suggestion. I'm thinking this could work well as a self-hosted ollama+notmuch setup...
firedrake_feed2024 paranormal fantasy, third novel of its series. Bunny Barrington, involuntary vampire and new-made police in a tiny supernatural town in Alaska, tries to work out who stole the gems that power the magical barrier.
cks_techblog_feedA long time ago I wrote up some basic customizations of Cinnamon, shortly after I started using Cinnamon (also) on my laptop of the time. Since then, the laptop got replaced with another one and various things changed in both the land of Cinnamon and my customizations (eg, also). Today I feel like writing down a general outline of my current customizations, which fall into a number of areas from the modest but visible to the large but invisible.
The large but invisible category is that just like on my main fvwm-based desktop environment, I use xcape (plus a custom Cinnamon key binding for a weird key combination) to invoke my custom dmenu setup (1, 2) when I tap the CapsLock key. I have dmenu set to come up horizontally on the top of the display, which Cinnamon conveniently leaves alone in the default setup (it has its bar at the bottom). And of course I make CapsLock into an additional Control key when held.
(On the laptop I'm using a very old method of doing this. On more modern Cinnamon setups in virtual machines, I do this with Settings → Keyboard → Layout → Options, and then in the CapsLock section set CapsLock to be an additional Ctrl key.)
To start xcape up and do some other things, like load X resources, I have a personal entry in Settings → Startup Applications that runs a script in my ~/bin/X11. I could probably do this in a more modern way with an assortment of .desktop files in ~/.config/autostart (which is where my 'Startup Applications' setting actually wind up) that run each thing individually or perhaps some systemd user units. But the current approach works and is easy to modify if I want to add or remove things (I can just edit the script).
I have a number of Cinnamon 'applets' installed on my laptop and my other Cinnamon VM setups. The ones I have everywhere are Spices Update and Shutdown Applet, the latter because if I tell the (virtual) machine to log me off, shut down, or restart, I generally don't want to be nagged about it. On my laptop I also have CPU Frequency Applet (set to only display a summary) and CPU Temperature Indicator, for no compelling reason. In all environments I also pin launchers for Firefox and (Gnome) Terminal to the Cinnamon bottom bar, because I start both of them often enough. I position the Shutdown Applet on the left side, next to the launchers, because I think of it as a peculiar 'launcher' instead of an applet (on the right).
(The default Cinnamon keybindings also start a terminal with
Ctrl + Alt + T, which you can still find through the same process
from several years ago provided that
you don't cleverly put something in .local/share/glib-2.0/schemas
and then run 'glib-compile-schemas .' in that directory. If I was
a smarter bear, I'd understand what I should have done when I was
experimenting with something.)
On my virtual machines with Cinnamon, I don't bother with the whole xcape and dmenu framework, but I do set up the applets and the launchers and fix CapsLock.
(This entry was sort of inspired by someone I know who just became a Linux desktop user (after being a long time terminal user).)
I have these (on my laptop) and perpetually forget about them, so I'm going to write them down now so perhaps that will change.
move-to-corner-ne=['<Alt><Super>Right'] move-to-corner-nw=['<Alt><Super>Left'] move-to-corner-se=['<Primary><Alt><Super>Right'] move-to-corner-sw=['<Primary><Alt><Super>Left'] move-to-side-e=['<Shift><Alt><Super>Right'] move-to-side-n=['<Shift><Alt><Super>Up'] move-to-side-s=['<Shift><Alt><Super>Down'] move-to-side-w=['<Shift><Alt><Super>Left']
I have some other keybindings on the laptop but they're even less important, especially once I added dmenu.
scalziwhatever_feed
See, isn’t that nice? Lowers your blood pressure all the way down, it does. We could all use that right about now.
— JS
hacker_news_daily_feedThe 10 highest-rated articles on Hacker News on April 21, 2025 which have not appeared on any previous Hacker News Daily are:
tedu_inks_feedThese things mean that despite Go having a GC, it’s possible to do manual memory management in pure Go and in cooperation with the GC (although without any help from the runtime package). To demonstrate this, we will be building an untyped, garbage-collected arena abstraction in Go which relies on several GC implementation details.
matklad_feedThis short example is worth pondering a bit if you are learning Zig:
fn f(comptime x: bool) if (x) u32 else bool {
return if (x) 0 else false;
}
const print = @import("std").debug.print;
pub fn main() void {
print("{} {}", .{f(false), f(true)});
}It is curious in three ways:
comptime parameters.
-> auto. When a generic
function is called, Zig first computes the signature. While doing so, it ignores function’s body.
Afterwards, the compiler separately checks that the signature is correct for the call site,
and that the body is correct for the signature.
alicebartlett_feedThis week I got 3 tonnes of topsoil delivered to the house so I can put it all in the veg plot. It’s steaming warm on the inside. On Friday I moved a whole tonne, and planted out the broad beans, peas and nasturtiums. This is absolute YOLO-chaos of course. You have to let it settle for a couple of weeks before you plant into it, but I don’t have a couple of weeks — I have now or I have never. Fite me, nature.
I read a magazine called The Fence the other day and in the back it had some quick-fire questions for one of the men from Rizzle Kicks. Here are the ones I can remember — you can send me your answers if you like. A little Easter treat. I’ve added some of my own too.
drmaciver_feedResuming habits
I took a deliberate break from the daily posting habit over the easter weekend. I think this was the right choice.
But now, I’m faced with a perennial difficult problem: Once you’ve paused a habit, you’ve lost inertia, and now you have to deal with the difficult task of starting again from a cold start.
What I’ve often found in the past is that this causes you to delay, and then the habit develops an ugh field, and this makes it harder and harder to start, and suddenly months have passed without you having touched it.
I think the solution to this is that whenever resuming a habit, even after an incredibly short break but certainly after a long one, you should deliberately do so in the lowest effort way possible.
Like this.
waxy_links_feedsmbc_comics_feed
Hovertext:
The other one of course is Schindler's List.
nwhyte_atom_feedSecond paragraph of third section:
‘All I ever seem to do is boring Latin.’
Soon after reading some of her father’s work, I got hold of his daughter’s best known book. The only work of hers that I previously remember reading is The Captains and the Kings, at least thirty-five years ago.
This is a short, swift, very sad story about a friendship across class and religious lines in pre-first world war rural Ireland, which then plays out grimly in the trenches. There’s a wealth of hidden sexuality and buried family secrets, and the politics of conflict which plays out as much in the internal tensions of the Irish troops as with the Germans. It’s very well done. You can get it here.
dotaturls_feedjwz_blog_feed
Dear Lazyweb, Seeking source for simple but believable OpenGL 3D fire and smoke simulation that does not use GLES / GLSL shaders, OpenGL 1.3 only.
firedrake_feed1959 suspense/thriller. Evelyn Manville was jilted, and went to work abroad. A few years later, back in London and broke, he runs into his old flame, now a widow with a very troublesome son…
domainincite_feedRemember January 20, 2025, about a thousand years ago, when Donald Trump was inaugurated for his second term as President of the United States? Remember how the dais at the Capitol rotunda was stacked with tech bros including Mark Zuckerberg, Jeff Bezos and Tim Cook, each of whom had authorized million-dollar donations to the Trump […]
The post Verisign gave Trump $100,000 first appeared on Domain Incite.
cks_techblog_feedDANE is commonly cited as as "wouldn't it be nice" alternative to the current web TLS ('PKI') system. It's my view that DANE is an example of why global DNS isn't a database and shouldn't be used as one. The usual way to describe DANE is that 'it lets you publish your TLS certificates in DNS'. This is not actually what it does, because DNS does not 'publish' anything in the sense of a database or a global directory. DANE lets some unknown set of entities advertise some unknown set of TLS certificates for your site to an unknown set of people. Or at least you don't know the scope of the entities, the TLS certificates, and the people, apart from you, your TLS certificate, and the people who (maybe) come directly to you without being intercepted.
(This is in a theoretical world where DNSSEC is widely deployed and reaches all the way to programs that are doing DNS resolution. That is not this world, where DNSSEC has failed.)
DNS specifically allows servers (run by people) to make up answers to things they get asked. Obviously this would be bad when the answers are about your TLS certificates, so DANE and other things like it try to paper over the problem by adding a cascading hierarchy of signing. The problem is that this doesn't eliminate the issue, it merely narrows who can insert themselves into the chain of trust, from 'the entire world' to 'anyone already in the DNSSEC path or who can break into it', including the TLD operator for your domain's TLD.
There are a relatively small number of Certificate Authorities in the world and even large ones have had problems, never mind the one that got completely compromised. Our most effective tool against TLS mis-issuance is exactly a replicated, distributed global record of issued certificates. DNS and DANE bypass this, unless you require all DANE-obtained TLS certificates to be in Certificate Transparency logs just like CA-issued TLS certificates (and even then, Certificate Transparency is an after the fact thing; the damage has probably been done once you detect it).
In addition, there's no obvious way to revoke or limit DNSSEC the way there is for a mis-behaving Certificate Authority. If a TLD had its DNSSEC environment completely compromised, does anyone think it would be removed from global DNS, the way DigiNotar was removed from global trust? That's not very likely; the damage would be too severe for most TLDs. One of the reasons that Certificate Authorities can be distrusted is that what they do is limited and you can replace one with another. This isn't true for DNS and TLDs.
DNS is extremely bad fit for a system where you absolutely want everyone to see the same 'answer' and to have high assurance that you know what that answer is (and that you are the only person who can put it there). It's especially bad if you want to globally limit who is trusted and allow that trust to be removed or limited without severe damage. In general, if security would be significantly compromised should people received a different answer than the one you set up, DNS is not what you want to use.
(I know, this is how DNS and email mostly work today, but that is historical evolution and backward compatibility. We would not design email to work like that if we were doing it from scratch today.)
(This entry was sparked by ghop's comment mentioning DANE on my first entry.)
sre_weekly_feedIn this final installment of the Scaling Nextdoor’s Datastores blog series, we detail how the Core-Services team at Nextdoor solved cache consistency challenges as part of a holistic approach to improve our database and cache scalability and usability.
I really enjoyed this whole series. Thanks, Nextdoor folks!
Slava Markeyev — Nextdoor
These folks analyzed a non-production incident like it was production, including retrospective analysis and lessons learned. Best part: they share the juicy details with us!
Joe Mckevitt — UptimeLabs
This one goes over several different models you can use to implement on-call compensation, with pros and cons for each.
Constant Fischer — PagerDuty
This article shows that MySQL’s CATS algorithm offers only a small performance gain over FIFO once deadlock logging interference is removed.
My jaw involuntarily opened when I saw the graph after they commented out the logging print statements.
Bin Wang — DZone
In this article, I’ll walk you through how we implemented chaos engineering across our stack using Chaos Toolkit, Chaos Monkey, and Istio — with hands-on examples for Java and Node.js. If you’re exploring ways to strengthen system resilience, this guide is packed with practical insights you can apply today.
The author does not appear to have a tie to Istio. This article has a ton of code snippets to help you get started.
Prabhu Chinnasamy — DZone
In this blog, we’ll look at three important facts about serverless reliability that teams often overlook. We’ll explain what they are, what the risks are of not addressing them, and how you can make your serverless applications more fault-tolerant.
- Serverless architectures don’t guarantee reliability.
- You do have control over serverless reliability.
- Serverless reliability practices can benefit all platforms, not just serverless platforms.
Andre Newman — Gremlin
This Golang debugging story is a really satisfying read.
The heap profiles were very effective at telling us the allocation sites of live objects, but provided no insights into why specific objects were being retained.
Ella Chao — WarpStream
Zoom had an outage this week when its domain zoom.us was temporarily blocked at the TLD level due to a miscommunication between its registrar and the TLD.
Zoom
scalziwhatever_feed
I got a new camera (a Nikon Coolpix P1100, review coming at some point), and one of the things it does really well is zoom in real close to far away objects. I tried it on the sunset today, and, yup, it got in real close. Enjoy.
— JS
john_naughton_feed
Majority Rules, OK?
”When fascism comes to the United States it will be wrapped in the American flag and will claim the name of 100-percent Americanism.”
Aretha Franklin | Good to Me as I Am to You
The Next Terrorist Attack
In a sobering essay, historian Timothy Snyder does some thinking ahead which prompted me to do some thinking about the past. His argument is that the way the Trump regime is hollowing out or controlling many of the most important parts of the American national security system makes the country more vulnerable to a terrorist attack. He then goes on to explore what might happen if such an attack were to materialise, and in particular how it might be exploited by the regime in Washington.
In just three months, the Trump people have made the unthinkable much more likely. They have created the conditions for terrorism, and thus for terror management. This is true at several levels.
Most obviously, they have debilitated the services that detect terrorist threats and prevent attacks: the Central Intelligence Agency (CIA), the Federal Bureau of Investigation (FBI), the Department of Homeland Security (DHS), and the National Security Agency (NSA). The CIA is a foreign intelligence service. The FBI is the federal police force. The NSA, which specializes in cryptography and foreign signals intelligence, is part of the Department of Defense. Homeland Security is a cabinet-level department that amalgamates a number of functions from immigration control through disaster relief and anti-terrorism.
Overall guidance over the intelligence agencies is exercised by Tulsi Gabbard, who is known as an apologist for the now-overthrown Assad regime in Syria and the Putin regime in Russia. The director of the FBI is Kash Patel, an author of children’s books that promote conspiracy theories, and a recipient of payments from sources linked to Russia. Patel plans to run the agency from Las Vegas, where he resides in the home of a Republican megadonor. The deputy director of the FBI is Dan Bongino, a right-wing entertainer who has called the FBI “irredeemable corrupt” and indulged in conspiracy theories about its special agents…
At one point he quotes a passage from his book On Tyranny:
Modern tyranny is terror management. When the terrorist attack comes, remember that authoritarians exploit such events in order to consolidate power. The sudden disaster that requires the end of checks and balances, the dissolution of opposition parties, the suspension of freedom of expression, the right to a fair trial, and so on, is the oldest trick in the Hitlerian book.
What this brings to mind is the Reichstag Fire of February 1933. Here’s how Wikipedia describes what happened. It was
an arson attack on the Reichstag building, home of the German parliament in Berlin, on Monday, 27 February 1933, precisely four weeks after Adolf Hitler was sworn in as Chancellor of Germany. Marinus van der Lubbe, a Dutch council communist, was the culprit; the Nazis attributed the fire to a group of Communist agitators, used it as a pretext to claim that Communists were plotting against the German government, and induced President Paul von Hindenburg to issue the Reichstag Fire Decree suspending civil liberties, and pursue a “ruthless confrontation” with the Communists. This made the fire pivotal in the establishment of Nazi Germany.
The decree nullified key civil liberties of German citizens. It was used as the legal basis for the imprisonment of anyone considered to be opponents of the Nazis, and to suppress publications not considered “friendly” to the Nazi cause. It was one of the key steps in the establishment of a one-party Nazi state in Germany.
Now of course history doesn’t repeat itself. But then the lesson we learn from history, as some wag observed, is that we never learn from history.
Yesterday’s Observer column:
Question: what are the eight most annoying words in the English language? Answer: “Your call is important to us … please hold.” But when you have turned into a gibbering wreck after 10 minutes of your valuable time have ticked away – intermittently punctuated by assurances that, while your tormentor is “experiencing high call volumes at the moment”, nevertheless your call is still important to him/her/it – you can take comfort in the thought that you are not alone. In fact, you belong to the majority of sentient beings in an industrial society like ours.
Thanks to a useful piece of market research, we now have an idea of the numbers of victims of this industrial practice – at least in the UK…
Larry Summers on what’s happened to the US
From a conversation he had with Tasha Mounk
I’m feeling like I’m part of some kind of Kafkaesque economic tragedy. I think the master narrative, the big picture here, Yascha, is that the United States is turning itself into an emerging or a submerging market. There are set patterns that we associate with mature democracies. There are set patterns that we associate with developing countries, for which some people would use the term “banana republic.”
In mature democracies, it’s institutions that dominate; in banana republics, it’s personalities that dominate. In mature democracies, it’s the rule of law that governs interactions between businesses and between business and government; in emerging markets, it’s personalities, personal connection, and loyalty. In mature democracies, the central bank and finance sits with independence relative to politics; in emerging markets, that is much more in question. In mature democracies, the goal is interaction, openness, and prospering along with the world; in immature democracies, in emerging markets, it is nationalist economic policies tied to particular interests.
The United States in a stretch of a few short months is transforming from being the United States to being something much more like Juan Perón’s Argentina–and that is being recognized by markets. It’s being recognized in the economy. It’s being recognized by people…
He’s also struck — as I am — by the number of people who are now using burner phones when going to the US when leaving or returning to the US.
It’s been the case for a long time—I imagine you’ve done it too, Yascha—that when an American businessman or journalist or government figure goes to China, they don’t bring their usual cell phone. They bring a burner phone, which they’re going to discard afterwards so that they won’t be hacked. I’ve heard a half a dozen anecdotes within the last five days about people feeling a need for burner phones when they come to the United States from other countries. I’ve heard more than one anecdote of Americans taking a burner phone with them when they leave the country so that they’re not at risk of having their regular phone searched by the American government when they return. So it’s not just a market-economic thing. We are being seen the way authoritarian countries are usually seen, and that’s not something I ever expected from the United States.
He’s also good on Harvard’s resistance to Trump’s demands.
This Blog is also available as an email three days a week. If you think that might suit you better, why not subscribe? One email on Mondays, Wednesdays and Fridays delivered to your inbox at 5am UK time. It’s free, and you can always unsubscribe if you conclude your inbox is full enough already!
hacker_news_daily_feedThe 10 highest-rated articles on Hacker News on April 20, 2025 which have not appeared on any previous Hacker News Daily are:
lunasorcery_feedIt’s always nice when I’m able to get the subject in focus before they fly away ^^
lemire_feed
In software, we often use key-value data structures, where each key is unique and maps to a specific value. Common examples include dictionaries in Python, hash maps in Java, and objects in JavaScript. If you combine arrays with key-value data structures, you can represent most data.
In C++, we have two standard key-value data structures: the std::map and the std::unordered_map. Typically, the std::map is implemented as a tree (e.g., a red-black tree) with sorted keys, providing logarithmic time complexity O(log n) for lookups, insertions, and deletions, and maintaining keys in sorted order. The std::unordered_map is typically implemented as a hash table with unsorted keys, offering average-case constant time complexity O(1) for lookups, insertions, and deletions. In the std::unordered_map, the hash table uses a hashing function to map keys to indices in an array of buckets. Each bucket is essentially a container that can hold multiple key-value pairs, typically implemented as a linked list. When a key is hashed, the hash function computes an index corresponding to a bucket. If multiple keys hash to the same bucket (a collision), they are stored in that bucket’s linked list.
Quite often, we only need to look at the keys, or look at the values. The C++20 standard makes this convenient through the introduction of ranges (std::ranges::views::keys and std::ranges::views::values). Let us consider two functions using the ‘modern’ functional style. The first function sums the values and the next function counts how many keys (assuming that they are strings) start with a given prefix.
template<typename map_type> auto sum_values(const map_type& map) { auto values = map | std::ranges::views::values; return std::accumulate(values.begin(), values.end(), typename map_type::mapped_type{}); } template<typename map_type> size_t count_keys_with_prefix(const map_type& map, std::string_view prefix) { auto keys = map | std::ranges::views::keys; return std::ranges::count_if(keys, [prefix](std::string_view key) { return key.starts_with(prefix); }); }
template <typename map_type> auto sum_values_daniel(map_type &map) { typename map_type::mapped_type sum{}; for (const auto &value : map | std::ranges::views::values) { sum += value; } return sum; } template <typename map_type> size_t count_keys_with_prefix_daniel(const map_type &map, std::string_view prefix) { size_t count = 0; for (const auto &key : map | std::ranges::views::keys) { if (key.starts_with(prefix)) { ++count; } } return count; }
When you do not have C++20, you can write the same functions using a more conventional style:
template<typename map_type> typename map_type::mapped_type sum_values_cpp11(const map_type& map) { typename map_type::mapped_type sum = typename map_type::mapped_type(); for (typename map_type::const_iterator it = map.begin(); it != map.end(); ++it) { sum += it->second; } return sum; } template<typename map_type> size_t count_keys_with_prefix_cpp11(const map_type& map, const std::string& prefix) { size_t count = 0; for (typename map_type::const_iterator it = map.begin(); it != map.end(); ++it) { const std::string& key = it->first; if (key.size() >= prefix.size() && key.compare(0, prefix.size(), prefix) == 0) { ++count; } } return count; }
Let us compare the performance. I have generated a large key-value map with a thousand keys (as strings) with corresponding integer values. All my keys begin with ‘key_’ and I ask my functions to count how many keys begin with ‘key_’ (that should be all of them). I use both an std::map and std::unordered_map data source. I record how many nanoseconds I need to spend per key checked. I use two machines, one with an Intel Ice Lake processor running at 3.2 GHz with GCC 12 and LLVM 16 as the compilers and one with an Apple M2 processor running at 3.5 GHz with LLVM 15 as the compiler. They do not use the same instruction set, so we cannot directly compare the number of instructions retired: we expect an ARM-based system like the Apple M2 to use a few more instructions.
Using an std::map I get the following timings and instruction counts:
| function | Apple M2/LLVM15 | Intel Ice Lake/GCC12 | Intel Ice Lake/LLVM16 |
|---|---|---|---|
| C++20 functional | 2.2 ns | 5.5 ns | 5.8 ns |
| C++20 loop | 2.0 ns | 5.5 ns | 5.8 ns |
| C++11 | 4.0 ns | 5.5 ns | 5.5 ns |
| C++11 with starts_with | 2.0 ns | 5.5 ns | 5.8 ns |
| function | Apple M2/LLVM15 | Intel Ice Lake/GCC12 | Intel Ice Lake/LLVM16 |
|---|---|---|---|
| C++20 functional | 46 instructions | 29 instructions | 49 instructions |
| C++20 loop | 41 instructions | 29 instructions | 49 instructions |
| C++11 | 58 instructions | 30 instructions | 30 instructions |
| C++11 with starts_with | 43 instructions | 30 instructions | 49 instructions |
Using an std::unordered_map, I get the following timings and instruction counts:
| function | Apple M2/LLVM15 | Intel Ice Lake/GCC12 | Intel Ice Lake/LLVM16 |
|---|---|---|---|
| C++20 functional | 1.1 ns | 3.8 ns | 3.8 ns |
| C++20 loop | 0.9 ns | 5.4 ns | 3.8 ns |
| C++11 | 0.9 ns | 5.4 ns | 3.8 ns |
| C++11 with starts_with | 0.9 ns | 5.4 ns | 3.8 ns |
| function | Apple M2/LLVM15 | Intel Ice Lake/GCC12 | Intel Ice Lake/LLVM16 |
|---|---|---|---|
| C++20 functional | 35 instructions | 10 instructions | 30 instructions |
| C++20 loop | 30 instructions | 13 instructions | 30 instructions |
| C++11 | 26 instructions | 14 instructions | 12 instructions |
| C++11 with starts_with | 32 instructions | 14 instructions | 30 instructions |
We find that iterating over an std::map instance is slower than iterating over an std::unordered_map instance.
Performance comparisons between Apple M2/LLVM15 and Intel Ice Lake/GCC12 systems reveal striking differences in C++ code execution. While functional approaches (e.g., C++20 ranges) can outperform traditional loops by up to 40% under GCC 12, using starts_with for string comparisons dramatically halves running time for std::map on Apple/LLVM15.
Despite small input sizes fitting in cache, Intel’s iteration performance is not compute-bound, achieving under 2 instructions per cycle (IPC), while Apple’s exceeds 4 IPC, highlighting its architectural advantage in optimizing data access patterns.
justin_mason_feed - Apache Iceberg Internals Dive Deep On Performancefiredrake_feed - The Tomb of Dragons, Katherine Addisonapnic_blog_feed - APNIC 60 Call for Papers now opentedu_inks_feed - FediMeteo: How a Tiny €4 FreeBSD VPS Became a Global Weather Service for Thousandscks_techblog_feed - We've chosen to 'modernize' all of our ZFS filesystemsapnic_blog_feed - IPv6 capability reaches 50% in the Asia Pacific regionapnic_blog_feed - The Asia Pacific region prepares for WTDC-25hacker_news_daily_feed - Daily Hacker News for 2025-04-22john_naughton_feed - Wednesday 23 April, 2025justin_mason_feed - Evertopmatklad_feed - Against Horizontal Scrolldomainincite_feed - Dot-brand actually being used to get deletedtalks_cl_cam_feed - Tue 13 May 16:00: Title to be confirmedsalo_linkspam_feed - Why an overreliance on AI-driven modelling is bad for scienceroads_org_uk_feed - Silver bulletsmbc_comics_feed - Saturday Morning Breakfast Cereal - Pastnwhyte_atom_feed - The Sontaransschneier_no_tracking_feed - Android Improves Its Securitydrmaciver_feed - Algebra and insightscalziwhatever_feed - The Big Idea: Heather Tracyscalziwhatever_feed - Today’s Adventures in Dentistrysalo_linkspam_feed - Maybe Persistent but Certainly Not Unique: On the Proliferation of DOIs in Open Access Book Publishidotaturls_feed - NIST uncertainty machine.justin_mason_feed - notes on using an LLM for personal email searchfiredrake_feed - The Vampire and the Case of the Baleful Banshee, Heather G. Harris and Jilleen Dolbearecks_techblog_feed - My Cinnamon desktop customizations (as of 2025)scalziwhatever_feed - The World’s Hectic. Have a Dog Picture.hacker_news_daily_feed - Daily Hacker News for 2025-04-21tedu_inks_feed - Cheating the Reaper in Gomatklad_feed - A Fun Zig Programalicebartlett_feed - Week 346: Topsoildrmaciver_feed - Resuming habitswaxy_links_feed - Primer, an introduction to oscilloscope musicsmbc_comics_feed - Saturday Morning Breakfast Cereal - Warghsalo_linkspam_feed - Ad Blocking 101 | Privacy Internationalexistential_comics_feed - Aristotle: the First Nerdnwhyte_atom_feed - How Many Miles to Babylon?, by Jennifer Johnstonxkcd_feed - Air Factsalo_linkspam_feed - Regrets: Actors who sold AI avatars stuck in Black Mirror-esque dystopia - Ars Technicadotaturls_feed - Things Zig comptime won't do.jwz_blog_feed - Firefiredrake_feed - Death Mask, Ellis Petersdomainincite_feed - Verisign gave Trump $100,000cks_techblog_feed - I feel that DANE is not a good use of DNSsre_weekly_feed - SRE Weekly Issue #473scalziwhatever_feed - Sunset 4/20/25john_naughton_feed - Monday 21 April, 2025hacker_news_daily_feed - Daily Hacker News for 2025-04-20lunasorcery_feed - Bird between the brancheslemire_feed - Streamlined iteration: exploring keys and values in C++20![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
branchandroot
Another outstanding Article.
Lets hope, as all return to work and school that it works and it does improve air quality (if only a little bit!)
A couple of annoying (to me) issues
1) All the signage on approach is nicely bracketed into the retaining wall, except the "Congestion charge / Camera sign" which is posted... clearly an after thought.
2) Both exits are immediately complicated as shown on your video, either by a multi directional roundabout with traffic lights or a merge/diverge junction... just as your eyes get use to the light again...and the tunnel lights to prepare you didn't seem to be on.
Did I also see on the Southern Approach the Congestion Avoidance signs obstructing the main junction sign?
Any how BRILLIANT... I should have kept up my desire to be a road engineer :)