The days between, in Sweden known as mellandagarna, are the days between Xmas and New Year's, part of an extensive ritual where nearly everyone takes off from work starting in the middle of December and continuing until the middle of January.
And to think the Swedes are still one of the most per-capita per-annum productive work forces on the planet. But it's true. Go figure.
We're in the 'days between' right now. So it somehow seems fitting with a walk down Memory Lane. This walk takes us back to the ancient year of 1998 and moves forward at a brisk pace.
15 April 1998: Chris Pirillo somehow discovers Radsoft and leads his biweekly newsletter with us. Our server in the UK almost melts down.
1 May 1998: Chris does it again, due to server overload two weeks before. And by now we have, in a panic, changed IPPs so things run more smoothly. Yours truly has burns on his palm from rubbing on the Microsoft Windows mousepad gifted by a girl at the yearly Stockholm IT expo.
With these downloads, we're pulling in 1,500 email inquiries per day.
Standing at the workstation in our upstairs office in Cherry Hinton, we open Netscape's mail client and hit 'New Messages'. This is on dialup. It takes about ten seconds for ten new messages to drop in.
So we click 'New Messages' again. Same thing. Ten new messages in ten seconds.
And so it goes. 'No way we're going to do anything about this', I tell Agneta. 'Let's go eat lunch.'
So we set out for our favourite watering hole on the country road to Newmarket. A nice place, very low ceilings, decor in beige like the much more expensive Pink Geranium, but the food’s way better and they have a new cheesecake every week.
Chris' IP Probe
Chris Pirillo and I get corresponding after a while. He wants us to try our hand at an application for him. The file etc/hosts was a big deal back then - it stored IP resolutions so the DNS didn't have to look them up. Resolving with DNS just about doubled access times, and this is very noticeable on dialup.
Chris had a utility that did this, but his utility was written in Visual Basic, was of course single-threaded, and consequently v-e-r-y s-l-o-w. So he asked if we could do something.
(Most time on DNS resolutions is the waiting, so…)
I was teaching NT synchronisation at the time in the UK. Cutler's NT is loaded with synchronisation methods. The brilliance of Cutler’s model is that threads in a wait state consume zero CPU. They're simply not scheduled. So the model is very efficient.
I started pulling stuff out of the NT toolbox and came up with this. Chris' IP Probe, a backronym for CIP.
https://radsoft.net/gallery/cip/
Let's look at the toolbar to get an idea of what this critter can do.
New, Open, Save. Self-explanatory, but remember that this is Windows, so 'New' means 'discard whatever you have in that window and start again', and 'Open' also involves getting rid of whatever's in the window.
Edit, Add. For editing current records and creating new ones.
Cut, Copy, Paste. Standard editing tools. Per row.
Lightning bolt. Honestly NFC. Can't remember.
Traffic light. This starts the resolution process and turns to amber if you're trying to stop it, or something like that, before halting on red ('stop'). Something like that.
(The actual listing shows a number of 'disabled' URLs - they 'loop back' on 127.0.0.1. They're provided by Silencer, a separate (but compatible) Bloatbusters utility.)
Under the bonnet - this was used for a class tutorial - CIP uses everything including the kitchen sink. It calculates a reasonable number of threads to generate, uses a semaphore to regulate their disposal, and uses a number of other highfalutin' tricks to make things as smooth and as fast as possible.
Chris even got his own version with his own mutt in the icon.
The comments on the bottom of that page pretty well sum it up. There's only one thing to object to there: CIP is very much a 'tray app'.
I have this little piece of geniality running on a 56k and WIN98 SE. I've been using it for a mere 10 minutes and have seen some nice improvements in the loading speed of webpages. Especially those running flash animation. I have no clue on how it works but it does the job. Websites are found faster and show up on your screen substantially quicker. The developers of this little program deserve one hell of a pat on the back.
My English is not excellent. I have a small PC (modem 28.8, 1.9 gig disk, 75 as processor) and I needed a software like this. I had Webcelerator and this CIP is 2 times faster! No Icon, no tray... Perfect! I went on pages with Flash4, it's unbelievable! I recommend to read the web page, what I'm going to do right now to set up at best this little miracle. Great!
(Something that not a lot of people caught at the time was that you had to check whether your DNS response turned up with multiple IPs. Dynamics are in play in such case and the distribution should be left up to the remote server.)
Letter From Hong Kong
Then one day, several years after that, when we're embedded near the RTP in North Carolina, we get a letter from someone in Hong Kong. He absolutely loves CIP but he has a problem. He uses those 'New' and 'Open' buttons a lot. Things really slow down, he says.
O RLY?
'How many items do you have?' we ask. 'We got about 2,000 here.'
'Oh I got about 20,000, 25,000', he tells us.
Whoah.
Global to Heap
We'd probably been using Microsoft's ubiquitous GlobalAlloc API. Even with 2000-3000 entries, freeing that many records wasn't noticeably time-consuming. But 25,000? Time to move over to Cutler's HeapAlloc API.
Cutler's HeapAlloc API is another brilliant piece of work by the Tribe. You start by creating a 'heap'. A heap is this thing where you'll put allocations. You keep the reference you get back when you create it, then use that reference when allocating. And, behind the scenes, the Tribe keep shifting things around to make it all very tidy. At the end, when you don't need it anymore, you issue a single call to free it all. One call. And it's all gone. Whoosh.
We updated CIP for Windows and sent a copy to our new friend in Hong Kong. But we were just getting into Apple's OS X, and a lot of people were still on dialup, so CIP became a prime candidate for porting.
Somewhere out there on the InterTubes there were a couple of forums for OS X developers. They were populated by the two types of developers, all with little experience in OS X, but some coming from a non-Apple platform and some coming from an Apple platform. Those in the latter group were at a stark disadvantage. They knew little about threading or anything like that. And they were generally a nasty crew. Even though they'd studiously ignored Steve Jobs in Redwood City, they acted like they owned him now. And even though their old 'MacOS' had jack-shit to do with OS X, they acted like they knew it all. The people coming from other platforms were generally friendly, enthusiastic, and helpful, whilst the Maccies could turn nasty and resentful on a dime. They resented Steve Jobs upending their precious little world, they resented being sent back to school, etc. All this was discovered over time - little was known at the beginning.
But I jumped into one of those forums. For I had a question. Namely: Did Unix or OS X have the equivalent of Cutler's HeapAlloc API?
A firestorm ensued. Goodness gracious. What's hard to understand when residing in the realm of the sane is that no criticism, even indirect criticism, of anything Apple is ever allowed. The thought that Apple should lack such an API, even if it's not yet been understood to be useful, is tantamount to treason.
One of the first comments was that no listing, anywhere, ever, could have as much as 4,000 records. (You can't make that shit up.)
A certain Swiss programmer, versed in old MacOS but not NeXT, who'd written to us earlier, boasting about his front end to an open source chess machine, wrote that we couldn't blame Apple for our own crappy code. Whereupon they found out that the sample code that we'd posted came directly from Aaron Hillegass.
Panic ensued, with one of the 'Landed Gentry' yelping: 'Aaron! Get in here! We know you're in here, lurking!'
But the cowboy of course never appeared.
And so goes it with Apple's wonderful 'Mac community', a stark foreshadowing of the woke bullshit that was to come. People lose their jobs because they criticise Apple in public. Websites get shut down. We've been all but cancelled by Apple media - any media company dependent on advertising revenues from Apple. You don't criticise Apple in a free country. It's as simple as that.
(There’s an old piece that we can’t find. It’s called ‘Tableview Blues’ or something similar. It tells a bit of the story.)
It Wasn't Good Then, It's No Better Now
They weren’t perfect at NeXT. They didn’t know everything. And some things only became apparent over time. Microsoft’s understanding of file management interfaces turned out to be correct. NeXT’s turned out to be somewhat wrong. (And let’s not get into Apple’s.)
But our own Xfile is the only Apple OS desktop file management utility that can survive stress tests. It alone was built the right way - borrowing on ideas from the CIP utility described earlier. (That’s Xfile in the video at the top of this piece.)
And one of the key things is disposing of memory quickly when you no longer need it.
Trying to do full listings is something neither Finder nor Path Finder can handle. Or you want to talk about expanding an entire hive in either of them? Don’t try.
They’re designed wrong.
https://rixstep.com/2/2/20100122,00.shtml
The Linux GUIs, which ape Windows in everything, have better file managers.
Try this as an experiment. From the 10.5 days.
Open either Finder or Path Finder (or both) and make sure you're in the 'list mode' - the one with the disclosure triangles to the left of the file items.
Option-click on /System. Then hold on for dear life. Path Finder understands a bit of what's going on - it will automatically stop things after a few levels are expanded. (The option-click - or option-right arrow with a selection - is an 'expand all' command.) Back to Path Finder in a minute.
The poor asthmatic Finder will just keep on expanding (at least on 10.5). This can go on seemingly forever. (10.5's /System has over 90,000 items, 10.6's /System has over 100,000.)
The best thing to do for Finder is to kill it - and to remember to never ever thereafter click the wrong disclosure triangle again.
Path Finder is a bit different - it stops the expansion after two levels. It can do this by responding to NSOutlineView's delegate method outlineView:shouldExpandItem:. It takes a lot of crunching - especially when you're supposed to be expanding as fast as you can - but it saves the day. At least temporarily.
The coup de grãce comes when the sorry thing (Path Finder now, not Finder) is finally finished expanding all 90,000-100,000 items to its two levels. You only need a few columns of data open on the right for this to be sufficient. Go to the lower right corner and click that wee innocent button to 'refresh'.
Kill Path Finder when you get tired of the exercise and/or when your laptop starts burning through your clothes and your Core 2 Duo starts melting your mobo.
In both cases - both Finder and its mutant friend Path Finder - the NSOutlineView represents an Achilles heel. This is not a bug and there is no workaround. It's the design itself that's flawed - and not of NSOutlineView itself but in the choice of this control for file management.
So what we did, natch, for Xfile was create a mutant for that all-important left column. That’s part of the reason ‘it just works’.
We got reports of an issue sometime after 10.4 Tiger. People reported crashes on startup - sometimes. It wasn’t until we’d upgraded ourselves that we found the reason. Apple had, on the sly, changed their ‘message pump’ model. With that fixed, Xfile hasn’t crashed a single time since. And that’s over fifteen years ago.
Apple’s latest OS iteration, when in beta, had reliable users reporting that Finder was so flawed, and so buggy, that its frequent crashes could force a cold reboot of the entire operating system. How is that even possible?
As seen many years earlier with an abortive update to Microsoft's Internet Explorer, where an errant '.' in the location bar could cause a BSOD, it is in fact possible when:
The code goes places it should never go (ie somehow escalates its privileges and crosses over into dangerous supervisor mode territory).
The system itself lets this happen.
And all this nonsense for a file manager.
At a systems programming course I held for British Aerospace in the UK we had one delegate who’d just returned from down under with a CompSci degree.
For his entire curriculum, he told us, they used beige box Apple computers.