Sunday, July 23, 2006

Hidden gem

Having been using fink for at least three years, I'm surprised it's taken me this long to discover the 'applesystemfonts' package, which lets you use the, um, Apple system fonts under X.  But I'm glad I did, as it's no longer "Times New Roman or nowt" in Scribus :-)

Tuesday, July 11, 2006

Irritating annoying irritationnoyance of the day

WebObjects generated components use a style like this:
    public Main(WOContext context) {
        super(context);
    }

But if you add a key using WOBuilder, then its accessors use the following style:
    public String username()
    {
        return username;
    }

Mmm...you can almost smell the consistency.

Friday, July 07, 2006

This year's WWDC t-shirt?


"'ad you not nailed it to iTunes, it'd be pushin' up the daisies!"

Thursday, July 06, 2006

No worries, mate!

Just got a brace of MacBooks, and with the .pvs linked in an earlier post OPENSTEP/Mach works on both with Parallels.

Wednesday, July 05, 2006

Swedish chef

Yes, I did write a swedish chef to C translator.  What of it?  My cousin was bitten by a møose once.

Thursday, June 29, 2006

Lest I forget

PIN END
 ___
/   |ooo|
-----
 bce

Wednesday, June 28, 2006

Subconscious operation

I just noticed that I've been using Xcode's codesense all day, despite the fact that I'm writing Python (albeit with PyObjC goodness).  I wasn't aware that that would work, but then I wasn't really aware that it was working, if you see what I mean.

Thursday, June 22, 2006

OS r us

I reckon I might be some kind of operating-system collecting nerd.  Having played with various systems on the Intel Mac at work through Parallels (NeXTSTEP, OPENSTEP, Rhapsody, Debian GNU/Linux, Solaris 10, DesktopBSD) got me wondering why I don't do the same at home.  So having fired up Qemu on the iBook, I'm now in the process of setting up the HURD.  There's a bit of a messy GRUB invocation involved before it will boot, but then everything works pretty well.

I'm just left wondering exactly what it is I'm going to *do* with my new-found HURD installation :-)

Monday, June 19, 2006

PVS file

Quite a few people seem to have issues with Parallels/OPENSTEP/Mach, so let's at least remove Parallels configuration from the list of unknowns.  Try this PVS file, see if that helps.  BTW, anyone having problems, what host system are you using?

Thursday, June 15, 2006

Use of vm_stat(1)

[N.B. I'll make heavy use of links to DarwinGrok in this post. However, I make no comment on the availability of darwingrok; it's set up on a test server which is unsupported, to test how well said unsupported server handles a Java app deployment mechanism which is also currently unsupported. It's been known to go down for days before I realise anything's wrong. However, if it turns out to be unavailable, just look at the end of the URL to find out which file I've linked to, and look for that on darwinsource. For sake of comparison between the two sites, darwingrok's source cache is based on 10.4.5.ppc.]

So, it seems a bit strange to be talking about vm_stat(1) when I recently posted that I'd been working on an alternative to it. But in doing so, as was rightly observed in the comments, I did learn a thing or two about the Mach VM manager and vm_stat is about the most verbose place in which it's encountered. And besides, I had a request :-)

Let's start by looking at some sample output:
Mach Virtual Memory Statistics: (page size of 4096 bytes)
Pages free: 18150.
Pages active: 129075.
Pages inactive: 79964.
Pages wired down: 34856.
"Translation faults": 216587934.
Pages copy-on-write: 3932355.
Pages zero filled: 85582532.
Pages reactivated: 1712668.
Pageins: 474657.
Pageouts: 77330.
Object cache: 478244 hits of 1592868 lookups (30% hit rate)


Just for comparison, here's the output of "free -p" shortly after that (those of you who haven't investigated free since my last blog post are in for a surprise):
total used free
Mem: 262045 244025 18020
Swap: 262144 171028 91116


I'll start by looking at physical memory, because that's a whole topic in itself which I've spoken about so many times I can cover in a self-contained glob. As an aside though, the pagesize is a compile-time setting, and is 4096 bytes on both architectures unless you go compiling your own kernel. If you do, remember that it's got to be a power-of-2 multiple of the hardware page size.

So, physical memory then. Which of the various pages that we're being told about are actually in physical memory? The active ones must be, and so must the wired ones (a page marked as wired may not be swapped, so by definition they're stored in physical RAM). Actually it turns out that the free and inactive pages are both in RAM too; anything that's been handed out to a pager isn't counted as an available page (but the kernel knows where to ask for that page, if it needs it back). There's no information on page counts outside physical RAM, which was why earlier versions of free(1) couldn't report on the available swap space. It also happens not to make much sense to ask how much swap space is available, because it can change without user interaction. Compare this with Linux: if you swapon(2) a 2GB partition and nothing else, then no matter how much swap space is currently in use, the kernel knows that there's 2GB of swap available. Anyway, that's an aside. Add up the number of free, active, inactive and wired pages, multiply by the page size and you should have a number familiar to you as the total amount of RAM installed in your Mac (or arbitrary other Mach OS box).

Next question, what is the amount of unused RAM? I get asked this frequently and my usual answer is: Heh :-). The free tool just looks at free pages, and top(1) goes for this approach too. Other people will say that it's the number of free and inactive pages. And they're right too, I think. Free pages are absolutely free, and have not been claimed by anything. Inactive pages are candidates for reuse, but do actually contain data. An example of when pages might become inactive is that a task is launched and loads a few dynamic libraries in, then quits. It'd be handy to keep those dylibs around just in case (try launching an app like OmniOutliner, quitting, and relaunching. Even on a poor man's metric like # of dock bounces, it was faster the second time, right?) but if the memory needs to be claimed elsewhere, it can be. So the amount of unused RAM is a nontrivial quantity, but if you mean "how much RAM is free" the answer is the free pages. If you mean "how much more RAM could I activate" then it's that in free or inactive pages. You'll have real performance problems if you've got few free pages and and lots of wired pages, because the wired pages can't go anywhere you've got that much memory less for user-space processes to swap in.

Okey dokey, let's have a look at the paging statistics next then. All of the following values are cumulative; they start at 0 at boot time and monotonically increase with (up)time.

The state of a page is known to the kernel in terms of a few variables:
  • Empty

  • Wired

  • Dirt (i.e. has the page been modified)

  • Current access privileges (none, read, or read/write)

  • Desired access privileges

There are lots of different mechanisms by which the state as described above can be modified, and each change of state triggers at least one "translation fault". These faults are handled by vm_fault(), so every time that's called the translation fault count goes up.

When a new Mach task is generated, it has an inheritance property which describes whether it starts with its own memory objects or whether it receives copies from its parent (as would happen with a fork(2) call). But rather than generating all those copies when the task is launched, the memory manager gives it shadow objects which refer to the original memory objects. The task doesn't receive its own copy until it tries to write to the shadow object, at which point that generates a copy-on-write fault and the local version of the memory is finally generated.

"Pages zero-filled" describes exactly its own purpose, I hope.

A reactivated page is one that shifts from being inactive to active, as in the example of re-launching an application given above. Reactivating pages means less going out to disk to recover the data, which means faster performance. Finally, a pagein occurs whenever a new page is requested from the pager (basically, whenever a peek or poke into memory that has not yet been dealt with occurs) and a pageout occurs whenever a page has to be handed off to a pager to allow another page to be created or paged back in.

The problem with all of the above information is that it's not presented in a way which I'd find useful :-). I'd prefer my RAM statistics to be presented in more comfortable memory units such as megabytes; this is analogous to the default output of df(1) being the 512-byte block. I don't care about most of the other VM statistics at all, because they're only important if you're debugging the memory manager or a pager. Rather than the number of pageouts since I switched the computer on, I'd like to know the number of pages currently out (i.e. the amount of swap use). There's a sysctl "vm.swapusage" which reports on swap usage, which I've used in free. The best way to get at swap information would be to interrogate the pager(s), not the kernel, and looking at Apple's I don't think there's currently a way to achieve that. But then there are a few things the dynamic_pager would do if I were world emperor that it currently can't; reporting how far off a high-water alert is, cleanly turning off paging to a particular directory, maintaining a single table of all the files currently in use by all the dynamic_pager instances. Anyway, remember what I said earlier about the amount of swap being changable; all we get from vm.swapusage is the amount of filesystem which is currently dedicated to swap.

One thing which can be achieved with the available information, and which vm_stat doesn't report (neither does free) but top(1) does, and which is incredibly useful, is the differential with respect to time. Go on, fire up top. Where there are the pagein and pageout figures, the little number in brackets tells you the increase since the last report. Now a graph of that over time would tell you what effect the system utilisation was having on the memory utilisation. That'd be sweet.

The final line of vm_stat's output is the object cache hit rate, which is the ratio of objects requested vs. those which were in active memory at time of request. Obviously, the higher the ratio, the fewer reactivations and pageins. On a bunch of different Macs I have access to, the hit rate is between 2% and 30%.

BTW, would you like some references? Here's the OSF Mach documentation (the Principles manual contains a good chapter on virtual memory management), Apple's Kernel Programming guide, and the paper describing Mach's virtual memory management by the guys at CMU.

Edit 2006-06-18 09:07 GMT: stopped claiming that free doesn't know about swap space. It does.

Tuesday, June 13, 2006

free as in, well, free(1)

The linux tool free(1) just reports how much memory is free (and how much is in use).  Simple, really.  There's no obvious equivalent for Darwin, though.  We've got top(1) which is overkill, and vm_stat(1) which is abstract to say the least.  I wrote a variant of free(1) for Darwin a while back, but never reported on the virtual memory (swap) usage.  Now I've found out enough about the Mach virtual memory manager to include that, there's an updated Darwin free available.  Excuse the manpage, I haven't updated it to track the new version yet.

I also found out enough about the Mach virtual memory manager to realise that there are a few things I want to change about the dynamic pager.  Without a degree in operating systems design, this might take a while...

Friday, June 02, 2006

NetBSD running on G5

Hurrah! NetBSD on the G5 gives us a potentially useful option for G5 servers when Darwin/BSD is inevitably phased out for the PPC...it'd be interesting if COMPAT_DARWIN comes along to the point where NetBSD can actually run WindowServer too.

I actually had a FreeBSD 7.0-SNAP up on my G4 tower a few months ago, hoping to help out with the port, but the kernel was so unstable that I couldn't get anything done at all :(. And I'm not as much of a kernel hacker as auserland hacker...

Tuesday, May 23, 2006

Parallels RC and OPENSTEP works!

I know that there've been mixed results in trying to get OPENSTEP booting on Parallels Desktop RC, but it Works For Me^{TM}. In case this is a particularly special piece of information, I upgraded directly from beta 4 (which I'd previously been using because 5 onwards were breaking the VBE driver). Both immediately after the upgrade, and following a reboot (I remembered that Parallels installs a kext, and thought it best to verify this), OPENSTEP boots up in glorious(?) multicolour.

Eww, although there's a graphics glitch displaying the "please wait until it's safe to turn off your computer" screen.

Update 2006-06-01 11:08 GMT: and so does RC2.

Thursday, May 18, 2006

New c.l.o-c alternative FAQ

So there used to be an alternative comp.lang.objective-c FAQ, but that never really got going.  I've been working on a new c.l.o-c FAQ listing with Mike Ash from Rogue Amoeba.  Why an alternative FAQ?  The official one is maintained by someone who refuses to take gcc ObjC and the various frameworks for it seriously - in fact anything which isn't his own (buggy, seldom-used) ObjC preprocessor.  We've just been a bit more accepting in what we talk about.

Tuesday, May 16, 2006

This so goes on my CV

Today's sysadmin skills involved rescuing some ALGOL code from an 8" floppy, accessed from CP/M on an RM380Z:

Amazingly although the disk hadn't been accessed in over a decade (though was stored in a Cool Dry Place^{TM}), everything worked first time.  The little jobby with a keyboard and built-in screen on the left is a Cambridge Z88, being used as an emergency VT52 terminal in this case.

Friday, May 12, 2006

Today's broken user interface award...

...goes to printing on OS X Server. You'll notice that you can add or remove printers via Server Admin: however, if you do add one via that route then the system will try and get the printer to emit its PPD at it. If the printer doesn't do this, then it's a "Generic Postscript Printer" and there's no way to change that.


Unless, of course, you log in at the console (Xserves make that so easy and snappy to achieve) and use the same Printer Setup Utility you'd otherwise have used on the client OS. You can then go back into Server Admin to set up whether a queue is quotad, set up sharing and so on.


Great, so far we've used two apps to configure our printer. Let's just add a third (although this does make sense), because you now have to go into workgroup manager to set up the per-user quotas for the print queues. Fine, and while we're here we'll use managed config to define which printers appear in which print panels. Only, we can't. Shared printers can't be added to the MCX print menus...even if they appear in my print menu.


This is so confusing it's almost fun...

Tuesday, May 09, 2006

Complex things possible

I was once told by a local Human-Computer Interface luminary (who claims to have in-depth knowledge of the way NeXTStep's UI works) that it's not possible in any existing GUI to have lines connecting separate windows.
Allow me to introduce a decade-old piece of scotch mist.

Wednesday, April 26, 2006

Parallels 2.1 beta 5

....will no longer boot OPENSTEP successfully, as far as I can tell.  At least, OS4.2 hangs out, in trying to load the VBE driver.  Reverting to Beta 4 works though.

If anyone has different experiences with Beta 5, could they comment here?

Tuesday, April 25, 2006

Too much confidence is good for you

So, Sun have a new CEO in the shape of President and erstwhile COO Jonathan Schwartz.  Schwartz joined Sun back when they acquired Lighthouse Design, a NeXTSTEP applications developer with an office suite for the platform.  Having slightly more bravado than sense, I sent this:
Hi Jonathan, I hope you do not mind receiving unsolicited personal mail.

Congratulations on your appointment as CEO of Sun.  Perhaps now would be a good time, in the spirit of openness and interoperability that Sun is famous for, and in extension of the success of the OpenSolaris project, to revisit that Lighthouse Design code in the basement with a view to releasing it under an acceptable open source licence?  Would you be able to also open up the source for Sun's OpenStep implementation, as a reference for Cocoa and GNUstep developers to see how it's done?  So doing would give Sun lots of 'good faith' points among Cocoa developers on the Mac platform, and further Sun's position at the front of the 21st Century's open source-centric software market.

By the way, Cocoa really is a great application development platform (as I'm sure you know with your OpenStep experience), and the combination of Apple's UNIX workstations with Sun's Solaris UNIX servers, both sporting a coherent and consistent Cocoa-based application stack and development environment would truly be a wonder to behold ;-).

Regards,

Graham.
I expect precisely nothing to occur...

Friday, April 21, 2006

The repair permissions fiasco

So, I'm far from the first to discuss the voodoo surrounding repair permissions, but I've never seen quite such a flagrant violation of penal code 1984 (It is a felony to pass off your own sloppy programming as a user configuration issue)...

Yesterday I bought some software from the Apple store (the developers and software shall remain nameless, but suffice it to say that it was an expansion pack for a game).  Came home in order to install it, that didn't quite work...I'd installed a (n official) patch for the original game, making its version newer than the version in the expansion pack.  OK, blow it away, and reinstall.

The installer is a PEF-based Stuffit thing (I didn't realise people still used either PEF or Stuffit), so doesn't know anything about the Authorisation Services API.  OK, switch to an admin user, run the installer.  Switch back to regular user, and run the game.

Erm, no in-game text.  On any of the UI elements.  That's a bit strange.  Let's check the FAQ on their website; apparently there's a particular file in the game which contains all of the text, if that's corrupted then no text.

It's not corrupted, it's just unreadable by anyone except the user I installed it as. Quick su-chmod-exit, try again. OK, in-game text. Asks for the licence key. Enter it, gets accepted....asks for the licence key. Enter it, check it, gets accepted....asks for the licence key. Couple more times, checking really carefully now that I've entered the correct key (no acc1d3nt4l el1t3ne5s). Definitely doesn't work.

Back to the website FAQ: I find the file where the licence key gets stored. In the app directory. Guess who it's not writable by? That's right, anyone...turns out that savegames get stored there too. So the application dir has to be writable by anyone who might want to play the game. Erm, that kindof sucks.

So I wrote to the company, explaining what I'd found, and got this response:

The fact that the game couldn't write to the nwncdkey.ini file probably means that OS X is having some sort of file permissions issue on your Mac. You might want to boot to your OS X CD, run the disk utility, and run "Repair Disk" and "Repair Disk Permissions" on your hard drive.


Repeat after me: WTF? Their software is installed by a Stuffit installer, not by Installer.app - this means there's no BOM, so there are no known permissions pertaining to that directory as far as the OS is concerned. If they got the permissions wrong on the way in (not unlikely, a number of my user accounts on various systems have custom umasks, and I bet Stuffit doesn't check that) then Disk Utility isn't going to do anything to fix their mistake. For completeness I ran a verify perms check, and indeed nothing came back (well yes something did, in a different area of the filesystem: I had myself chmodded an area of /Developer but last time I checked, Carbon games don't make heavy use of the developer documentation). But the Cult Of Mac is strong, and its idols shall not be slighted, so running repair permissions is obviously the fix in this situation. Perhaps I should zap the PRAM, start with extensions off, rebuild the Desktop and sacrifice me a goat while I'm at it.

I'm going to write "a polite response" to this particular company's support desk, explaining how repair perms works, why perms errors (in the BOM-perms-don't-match-filesystem-perms sense) were not - could not have been - the cause of my issue and requesting that they switch to an installer platform which, well, understands the Mac OS X filesystem. And that they start writing user-specific data like savegames and so on into ~/Library/Application Support/.