Monday, December 18, 2006
Monday, December 11, 2006
Copyleft
I no longer even own all the code I write in my spare time, on my own equipment. :-)
Thursday, December 07, 2006
TGD - John on acid
I'm about halfway through my OmniOutliner document of notes made while reading TGD now....
There's a paraphrase in TGD saying that if the epistles of Saint John the Apostle represent John on pot, then the book of Revelation is "John on acid". What makes this observation more interesting is that it may be literally true - or at least closer to the mark than the quip may at first appear.
There was a documentary by Tony Robinson on Revelation and the End of Days, called The Doomsday Code. The - frankly worrying - sentiments of some of the extreme christian right in America are the same as described in TGD; namely that the Middle East instability [inclusive] or global warming or the 2005 hurricane season herald the proximity of the rapture, and that rather than doing anything to help those caught up in these situations we should be actively encouraging the Apocalypse, and xtians should let their heathen friends know that, well, they told us so. In the course of describing modern End Timers, the documentary showed us where Revelation was penned.
There are tens of apocalyptic visions penned by post-Christ Jewish theologians, but only one has been included in what is now termed the canon of the New Testament. That book of the revelation of Jesus Christ to his disciple, John (which isn't accepted by the Eastern Orthodox as part of the divine liturgy) was written on the Greek island of Patmos sometime in the second half of the first Common Era century, and details two visions experienced by the John of the title (whether or not he be apostolic) while on the island. The other key fact to remember about Patmos is the abundance of fly agaric mushrooms, especially around the mouth of the cave of revelation.
TGD - begging the question
Many of the arguments pro-religious belief presented in TGD would form a nice corpus in a dictionary of critical analysis for the entry on "begging the question". However, Prof. Dawkins studiously avoids the phrase despite its rampant, repeated and some would say wanton applicability, and if I were ever asked to predict why (I haven't been, I just like the sound of my keyboard) I'd say that it's because the phrase is so frequently misused on both sides of the Atlantic ocean.
Indeed, when I was taught the phrase, the example I was given is that of proof of the existence of god: we can see the results of god's creation all around, therefore god created them, therefore god exists. That's about as short - and absurd - an example of the technique as can be demonstrated. In ordinary speech, people use the phrase "begging the question" to mean: the argument I have just heard is incomplete or fallacious, because it appears not to answer (or to directly raise) the following problem. Where the problem was not explicitly part of the argument. However, the meaning of the word "beg" should not be overlooked, and is key when calling an argument out as one that begs the question. In fact, it isn't the argument that begs the question: it's the conclusion which does so, and the conclusion is begged from the question. In other words, the meaning of the phrase is that an argument draws to a conclusion which already existed in its assumptions or axioms. My Latin isn't very good, but I might describe this kind of argument as QEQ - Said that which was said.
Let's look at, for example, the argument for the existence of purgatory (it's quite near the end of TGD which makes it easy for me to remember). The conclusion is that purgatory exists, and the justification is that people everywhere are praying for the souls of the dead because they believe that the souls are in purgatory, and surely prayer isn't pointless. I hope that going all slanty made the point in the argument where the conclusion was codified apparent - the reason we can be sure that purgatory exists because we are sure that purgatory exists. QEQ.
I realise that this had little to do with the thesis of TGD, but to beg the question is a cool phrase which should be used less, but with higher accuracy.
Tuesday, December 05, 2006
TGD - I'm not dead yet!
The title of this article is a quote from Monty Python and the Holy Grail - the body cart comes around the town to collect up the (presumably leprous) corpses, and along comes John Cleese with his ninepence and his cadaver. The only problem is that actually the corpse feels rather like he might pull through.
This is a situation which relates to the ever-decreasing sphere in which divine intervention can be invoked, such reduction being due to scientific advances. There are known cases where people have been thought dead and have "got better", or have been ruled out a long time before the actual event of their dying. As an example, a death through epilepsy can often cause the unfortunate to become unresponsive long before becoming dead, so that 'miraculous' events such as the body remaining warm for days after the death are recorded. While the meaning of the term "dead ringer" has nothing to do with the 19th Century safety coffin, such devices did exist. The whole purpose behind holding a wake was the hope that life might return to the deceased - because sometimes it did. Various poisons can lead to the body attaining a death-like state, while in fact the subject remains living; perhaps the most famous of these is TTX (the toxin carried by pufferfish); supposedly used by Vodou practitioners to create zombies. A good explanation of that is Ghosts, Vampires and Zombies: Cinema Fiction vs Physics Reality. The case of Wilfred Doricent, described in the arXiv paper, is recent: he died in March 1988 and was no longer displaying many of the symptoms of death by September 1989.
The point here is that there are modern cases where death is misdiagnosed. Given the modern equipment, techniques and advances in both theoretical and practical knowledge, one would assume that the rate of misdiagnosis of death has decreased over time (and in a related vein, the number of conditions which terminate terminally has decreased, too). How can we know, without even contemporary doctors' notes, whether a particular resurrection event was miraculous or just a case of mistaken interment? Pufferfish aren't particularly common around the middle East so perhaps Lazarus wasn't eating fugu but he may have had epilepsy, tetanus or a number of other conditions which were confused with fatality.
Returning to the parenthesis on terminal conditions, above, one can argue by progression that if there are fewer "terminal" conditions now in (kindof) 2000 than there were in 1800, then assuming no catastrophic dark age reversion there will be even fewer in 2200, fewer still in 2400 and so forth. So the ability to "miraculously" recover from a terminal condition should decrease over time too; not because the rate of miracles is decreasing but because the ability to explain the circumstances is increasing.
TGD - religion leaving the gene pool
If there was some kind of selection pressure which favoured, even in a roundabout way, predisposition toward religion, and if sexual selection naturally favours successful people (or those who exude success, anyway), then we have a (not too serious) quandary. Because the most high-profile adherents, the most apparently successful at religion, are often the clerics. Who are frequently celibate (or at least, are required to be: please do read the dictionary definition for nepotism though).
TGD Responses - selection for religion?
I'm going to do a few articles based on The God Delusion by Richard Dawkins. As this is a bit of a departure from the usual content of yon blog, I'll be sure to prefix the titles of each with TGD so that those so inclined can filter for/against them in their syndication readers. The articles will be a mixture of prose agreeing with parts of the book, arguing against parts, questions which the book made me ponder and for which I have no answer, and quips (I haven't yet decided whether to group these together or allow a post for each) answering specific ideas or even sentences. My only hope is that all of the articles - and the processes leading to their inception - are critical, in what I hope is the obvious intention of that word. Needless to say, I'll leave the punchline - my opinion of the binding theme of TGD and any changes it has had or not had on my views - until the final article.
I'll start this series by looking at his idea of religion having its roots in a "misfiring" of some advantageous trait which has been selected for. I would ask this question: why should that be so? In his description of evolution he explains the idea of a neutral mutation — one which overall has no positive nor negative effect on the fitness of the genome, but which can nevertheless become dominant (or at least prevalent). For instance, human earlobes either join at the bottom of the lobe to the face or they do not. I don't understand there to be any fitness advantage in either, yet at some time the allele(s) associated with earlobe stuck-to-facedness have, I presume, diverged from a common variant. Why couldn't theism be just one type of earlobe? What if the mean fitness of religious nonreligious "phenotypes" (I expect that's the wrong word…perhaps theotype is better) are not significantly different, and religious behaviour just happens to arise and have become common? I suppose there may be an answer to this, although if there is it wasn't presented in TGD.
There is a related argument in the book which I don't buy. In dismissing the idea of a straightforward selection pressure for evolution, Prof. Dawkins presents the Gedankenexperiment of a tribe who worship a war god, and whose religion dictates that they should go and fight the neighbouring tribes, in the course of which they acquire of course the resources of those other tribes, and propagate their seed further. The argument presented against this is that for any one individual it is more efficient to hang back while their companions do all the fighting, then to share a part in the spoils. Therefore the society would collapse, as no-one would want to do any fighting. This rather squarely misses the point of, for example Freya's D.Phil. thesis and the concept that a population based on cooperation (in this case the war-tribe, in Freya's case a biofilm) can in fact tolerate some fraction of "cheats" and that there is a stable state where proportion of cheats is non-zero but the population still thrives. Consider the British benefit system — some non-zero amount of money is spent on targeting benefit fraud, which keeps the fraud not at zero but at some level which both can be afforded and which leaves the spending justified. In fact, a canny war-tribe member would try to weasel their way out of the line of combat by arguing (rightly or otherwise) that they alone have some property that the average combatant does not possess, and would refer to the weaseling as "promotion" or officership.
Another potential explanation is an interpretation from Terry Pratchett, Jack Cohen and Ian Stewart's Science of Discworld series. In it, they describe the idea of "lies-to-children" in which various layers of falsehood are peeled back as a child becomes ready to accept the more complex, yet more accurate, explanations. What if this idea is extended to become lies-to-protoscientists? What if, as well as a moral Zeitgeist, there is a consensus of acceptable sophistication? So a society initially agrees that each tree, waterfall, star, planet etc. must have its own god, then finds that it can accept the god-of-concept ideas of the more recent polytheistic disciplines, then agrees that a single god, while more complex a solution, is palatable, all the while science is also working to provide the ultimate in sophisticated solutions: that which is born out of simple concepts, but which can be applied to give predictable and verifiable results.
Wednesday, November 29, 2006
e[Mac]s keyboard
Friday, November 17, 2006
Blasty from the pasty
Yesterday I had a little problem with the iBook, which was saved by some age-old knowledge from before the OS X release. Almost.
When I use an external USB mouse, I disable the internal trackpad. Now I wanted to take the computer into a different room in order to watch some TV (on the EyeTV), so unplugged all the USB stuff, plugged the EyeTV in....and the trackpad was still disabled. Now, a lesser person would have gone back into the original room, plugged the USB mouse in, and used it (or taken it with them). An equally lesser, but richer, person would own a Bluetooth mouse. But not I! I remembered a trick from the NeXT days, and returned my computer to trackpad-workingness in but a jiffy. Three jiffies.
The first thing to do is to log out - "as any fule kno" this is done from the keyboard by Command-Option-Shift-Q. At the loginwindow there still wasn't any trackpad activity, so I thought restarting the WindowServer might fix that. This is done by entering ">exit" as the username in loginwindow. Sadly, no joy. My next thought was to drop to a text console, unload the trackpad's driver kext, reload it and see what difference that made. Doing so requires entering ">console" as the username. However I decided not to pursue this as >console has been sadly broken throughout 10.4, and on some systems was broken in 10.3 too. It's annoying as occasional glitches where the graphical environment is FUBARed could probably be fixed in console mode. But this left one course of action. Enter ">restart" in the loginwindow and wait...
I should probably point out that I waited for the reboot longer than it would have taken to retrieve the mouse. But at least I wasn't defeated.
Wednesday, November 08, 2006
One reason multiple inheritance sucks
So I'm reviewing a book, and it happens to cover the way method lookups are performed in a particular language's object model. I'm not going to say what the book is because it's not relevant, and nothing discussed here has anything to do with that publication. But reading it made me realise just why even when the language allows it, my internal pain aversion system will always skip multiple inheritance as an option. Consider this Python code (with built-in python shell promptness):
>>> class One(object):
... def doStuff(self):
... print "One.doStuff()"
...
>>> class Two(One):
... pass
...
>>> class Three(One):
... def doStuff(self,x):
... print "Three.doStuff(%d)" % x
...
>>> class Four(Two,Three):
... pass
...
>>> class Five(Three,Two):
... pass
...
>>> a=Four()
>>> b=Five()
so, what is the method signature of a.doStuff() and b.doStuff()? Well, it depends entirely on the order in which the class hierarchy gets inspected, Python does things by leftright-bottomtop so both a and b inherit Three.doStuff(self,x). In the case of a, Four doesn't have doStuff() so it looks at Two, which doesn't so it looks at Three and finds it. However, Python also has another lookup mechanism (which was the only one available pre-2.3 or possibly 2.4, and can still be triggered by omitting the "object" superclass from One) which is left-bottomtop-right, so a inherits One's doStuff() and b inherits Three's. In the words of Peter Cook, that could confuse a stupid person...
So luckily Objective-C avoids this problem by only having an inheritance tree, and we can't add method implementations via protocols either. And my specific example is moot, because -doStuff and -doStuff: are different selectors.
Friday, November 03, 2006
if ([self isKindOfClass:[whore class]]) [self promote];
Monday, October 23, 2006
We're off to see the Wozzard....
So, they said that Steve Wozniak would be there, and that he'd be signing books. So I took my book to be signed. That wasn't wrong, was it?
N.B.: yes, I bought a copy of iWoz and had that signed too. But the iBook is cooler ;-)
Saturday, October 21, 2006
I feel sorry for...
Sunday, October 15, 2006
Best. App. Evar!
So, why hasn't it been made yet? Come on, you lazy coders! The hardest bit would be the brain-computer interface, but there are example I/O Kit projects on the ADC site.
Monday, October 09, 2006
A spot of ultravaiolence, my droogs?
So, with a recent cash injection and an ADC hardware discount coupon both burning a hole in my pocket, as well as a desire to crack on with the ObjC 2.0 work on the c.l.o-c FAQ; it seems an Intel Mac and a copy of some virtualisation tech are in order. The question becomes simple: do I jump now, and get a MacBoko or MacBoko Pro, or do I hold on in case the Core 2 Duo variants are around the corner? Or I could even combine the two and get an iMac, keeping the Vaio for corridor worrying? I'm tempted by the "buy now" approach, as I actually need^Wwant a new system now, not in January...
Wednesday, October 04, 2006
Init dead; no casualties
An interesting time if you happen to be an aging startup superserver; Ubuntu are to replace init with Upstart from "Edgy Eft". That's quite cool, the model in which everything needs to always come up and always in the same order put forward by both styles of init is long dead, and the system V style of init is particularly cumbersome (though someone at Apple clearly liked it, as they had both SystemStarter and watchdog in the past).
I'm actually both surprised and pleased to see a Linux distribution making the jump here, I've always considered them fairly conservative (apart from in throwing any old free package into the distribution; I mean from a "core stuff"[*] perspective. I know that FreeBSD attempted something with launchd last Summer, but the latter has too much mach_init magic which can't cleanly be excised.
[*]now Apple are going to start a CoreStuff API and sue me. Fantastic.
One thing from TFA saddened me (though I really ought to have expected it), and that's this:
To avoid reinventing the wheel, we first looked at how much effort it would be to use of modify the existing replacements to be able to do this. Sun SMF and Apple launchd were immediately ruled out due to licence issues. It was important for us that the solution be uncontroversially free so that other distributions might adopt it; many had already rejected these for GPL incompatibility reasons.
Yes, it's the standard "GPL first, utility second" argument. I have a fairly strong opinion here: if something works, and I can see that it works, and it works better than any alternative I have access to, I'll use it. That's why I'm posting this from OmniWeb on a Mac, despite [i]there being free web browsers around, [ii]my work involving Linux. I prefer to have access to the source, except where that involves a retrograde step in terms of utility. Anyway, I digress; the point here is that launchd is released under (well, depends which version you look at) the Apache licence, and OpenSolaris under the CDDL; both of which are free software licences, but are incompatible with the GPL on technicalities. That's nice if you really do want to split hairs, but GNU/Linux (if you like) is supposed to be a usable OS so shouldn't utility come first? Admit it, how many of you have uninstalled kaffe or gcj and are using the Sun jdk? More to the point, how many of you are using Apache (which is also released under the Apache licence)? It's time to get over it. I propose that if you want to work on a pure GNU system, you should indeed work on a pure GNU system and hie thineself over to the HURD mailing lists. I fully intend to, if a gobbet of spare time makes itself available soon...but more because I think the HURD's cool ;-).
Wednesday, September 20, 2006
GAH!
Friday, September 15, 2006
WikityWebsite
You may be interested in some source code.
Wednesday, September 13, 2006
Parallels may, in fact, be completely dumb
tyr:/Library/Parallels root# ls -l
total 16
-rw-rw-rw- 1 root wheel 40 Sep 12 15:52 .dhcp.en2
-rw-rw-rw- 1 root wheel 0 Jul 4 17:10 .dhcpd_configuration
-rw-rw-rw- 1 root wheel 0 Jul 4 17:10 .parallels_common_options
-rw-rw-rw- 1 root wheel 100 Jul 4 22:44 .parallels_license_2.2
Sunday, September 10, 2006
switching CPUs
I can waste a lot of time if an interesting question gets posed. The most recent culprit was Chris Ridd in uk.comp.sys.mac, wondering aloud if threads ever got shuttled between CPUs on a Mac. Here we go...
It certainly looks like the conditions for doing this through the API can never be met on a real Mac running xnu. All you can do is assign a thread or a task to a processor set, but not a single processor. Unless that processor happens to be the only member of a processor set. Thing is, on xnu there is only one processor set, which encompasses all of the processors. So there's no way to force your threads onto a particular processor in Mac OS X. The lack of multiple processor sets means that you couldn't have a box with, for instance, both a G5 and a Xeon running a single kernel, which is kindof sad in a way. It won't get missed, though.
It doesn't even look like there's an API to work out which CPU you're currently on. What you can do is get the port for the default processor set (the only processor set), then by manually furtling with the structs you can iterate through the CPUs (or you can ask your own mach_host_port for an array of the CPU ports). For each CPU you can get the thread queue, which you could compare with your own thread to see if it's the same. That'd be tedious, but would get the required information. Once there though, I don't see any way for the thread to move between any queues except the idle and run queues on its 'native' processor; so I think the answer is that no thread ever migrates between CPUs. I may be wrong, and am happy to entertain any information to the contrary.
Unsurprisingly with no API to do what I want, little useful code came out of this exercise. I found out how many cpus the system has, but that's it really.
Update 2006-09-10 16:25 GMT - I hate this not knowing, so I decided to play the Ask Amit card ;-).
Monday, September 04, 2006
OPENSTEP/Parallels/VirtualPC/Spam
I've been trying to reply to bbum's post, firstly my comments were removed as spam, then he approved them by hand, now they're gone again. So here they are again.
I wrote [in this blog] much about my experiences getting OPENSTEP/Parallels working. Networking's fine but slow, graphics is good, mouse is a bit jumpy. I was talking at WWDC with people who say that Microsoft VPC no longer supports OPENSTEP/Mach. You definitely can't boot native on a new Intel Mac, as there's no SATA driver (and no way to use a boot floppy).
BTW, the floppy images are available over HTTP from Apple: OPENSTEP floppyimages
Sunday, September 03, 2006
ObjC FAQ update
Thursday, August 31, 2006
Comment word verification
Wednesday, August 23, 2006
Better clear the office out, can't remember what the carpet looks like
Of course this has been brewing for a few weeks, but I wanted to make sure my eyes were crossed and my teas dotted before letting anyone else know. Especially on the big scary intarwebs.
New website
Saturday, August 19, 2006
Hell hath frozen over
Tuesday, August 15, 2006
WTF is a Veo?
Saturday, August 12, 2006
NeXT Luminaries
Here I am, trying to look as important as the bunch of people around me. Right bunch of luminaries they are too, but I won't spoil the surprise of trying to guess who everyone is. If you know, comment ahead. Points for recognising someone's hat without recognising the person themselves will not be awarded. All quite large, 2016x1512 JPEGs. Thanks to Ken Tabb for snapping the button.
Friday, August 11, 2006
Report from WWDC
You're not going to see many discussions of talks from the conference, and this is no exception. Beyond the keynote, the only talk I can discuss is one I went to today, which was a public talk by Amit Singh (author of the most excellent Mac OS X Internals, which I didn't bring with me in case the plane tipped over in flight) in the Apple Store across Market Street from the conference.
Amit's presentation style is as engaging as his writing, and he didn't let the wrath of the demo gods spoil what was an engaging talk. While he was necessarily light on substance (he couldn't just read the book out, or we'd all still be in there) the demos gave indication of what is possible with Mac OS X and the material discussed in Internals. In terms of OS X mentality we're of the same stable, although he clearly knows much more about the system than I could ever lay claim to.
I got to ask him what the best and worst features of Darwin are in comparison with other UNIX. His answers were interesting and enlightening; best is the pragmatism Apple engineers use in generating new API or taking from existing code; worst is the poor approach to open source taken by Darwin. I couldn't agree more.
Tuesday, August 08, 2006
Greetings, Jobs-pickers
It seems to be de rigeur to provide a rundown of the top 10 Leopard features from yesterday's Stevenote, and as I can neither sleep nor be bothered to read sample code, it's a bandwagon I'm happy to jump on, albeit in my own cynical style. Cue the Fluff Freeman voice effects...but first, let's look at the hardware. Amazing. The amount of space they've freed up in the cases is stonking, so both the Xserve and the Mac Pro now actually have Pro-level expandability.
- 64-bit top to bottom. What, you mean there are programmers out there too lazy to separate their workhorse from their presentation? Apart from convenience, and a bigger OS+app footprint, I don't see that we gain much here. Now we have four Mach-O architectures, and the system libraries have to implement all four.
- Time Machine. I'm assuming that the BBC wouldn't let Apple license the Doctor Who theme music. Because that time vortex looks just like, well, the time vortex, and every time I saw that UI I wanted to sing the theme (sorry, Back To The Future, you lose). On the other hand, this is a sweet sweet feature and I can't wait for Leopard GM so I can dare to switch it on...
- Ship the whole package. I can't remember nor be bothered to look up what Steve said but it was something along the lines of you now get all the apps with the OS. I for one cannot wait to pay £69 for Leopard and receive my free copy of Shake.
- Spaces. About. Blinking. Time.
- Spotlight enhancements. See my comments on Spaces ;-). Spotlight in Tiger was just plain broken, hopefully they've sorted this out properly. After gratuitously faffing about with the Mail UI in Tiger, only to make searching even worse than before, hopefully Maileopard will actually be able to find a mail I write. Networked spotlight definitely is most welcome.
- Universal Access. The new voiceover capability is indeed much better than anything available in a current mainstream OS, sounding less like Steven Hawking (which is I think the same synthesised voice as the Amiga) and more like Davros. But making this a Top 10 keynote feature? I smell government contract compliance...
- Mail features: stationery, notes, to dos. Oh. My. Gods. Would someone please build a partition wall between Steve's office and the UI team, he keeps suggesting stuff to them. Why would I want this? Why would I have an OS on which I can stably run multiple apps, and write myself little notes on Stickies widgets, if I'm then going to fold all of the functionality into the Mail application? Can't I use iCal or OmniPlan for my ToDos, like they're designed for? And stationery; I hate to shout, but HTML MAIL MUST BE BANNINATED FROM THE INTARWEBS.
- Core Animation. As my good bud Ken pointed out, this doesn't seem to be anything you couldn't previously do with Tiger classes such as NSAnimation or NSViewAnimation, but maybe it's a snatch easier. The thing is, judging by the reactions of the Americans in the crowd, I've got a feeling this is going to be the "Web 2.0" of the Mac UI. I feel violated.
- Dashboard features: dashcode, web clip. Oh great, now we get to see thousands upon thousands of derivative widgets. And web clip could've been so much cooler; it looks to me like it's "render this whole web page, and display a punch-out with the following CGRect". What it should be is "load this web page, and render this section of the DOM".
- iChat++. I don't really like tabs in Mac apps (AppKit does multiple windows per document, not multiple documents per window, as I said to John Gruber in response to his predictions, and the HIG deprecates trying) but I'm glad they went with OmniWeb-style tabs instead of Safari-style tabs. The ability to stream an app's view over iChat is absolutely fantastic though, this really is enterprise-level IM (regardless of the Photo Booth-style effects). I'm looking forward to taking a photo of my office to use as my iChat backdrop wherever I am, too :-)
So there we go. Sorry to sound incredibly underwhelmed, but there it is. There's been plenty of juice in the WWDC but not much of it came from the Stevenote.
Friday, August 04, 2006
See you at WWDC...?
Fellow attendees: don't know what my movements will be, except that I'll be at the Thirsty Bear on Monday with some WebObjects people, but I'd like to meet as many of you as possible so please add onetrueleeg at mac dot com to your iChat list! :-)
Sunday, July 23, 2006
Hidden gem
Tuesday, July 11, 2006
Irritating annoying irritationnoyance of the day
public Main(WOContext context) {But if you add a key using WOBuilder, then its accessors use the following style:
super(context);
}
public String username()Mmm...you can almost smell the consistency.
{
return username;
}
Friday, July 07, 2006
Thursday, July 06, 2006
No worries, mate!
Wednesday, July 05, 2006
Swedish chef
Thursday, June 29, 2006
Wednesday, June 28, 2006
Subconscious operation
Thursday, June 22, 2006
OS r us
I'm just left wondering exactly what it is I'm going to *do* with my new-found HURD installation :-)
Monday, June 19, 2006
Thursday, June 15, 2006
Use of vm_stat(1)
So, it seems a bit strange to be talking about vm_stat(1) when I recently posted that I'd been working on an alternative to it. But in doing so, as was rightly observed in the comments, I did learn a thing or two about the Mach VM manager and vm_stat is about the most verbose place in which it's encountered. And besides, I had a request :-)
Let's start by looking at some sample output:
Mach Virtual Memory Statistics: (page size of 4096 bytes)
Pages free: 18150.
Pages active: 129075.
Pages inactive: 79964.
Pages wired down: 34856.
"Translation faults": 216587934.
Pages copy-on-write: 3932355.
Pages zero filled: 85582532.
Pages reactivated: 1712668.
Pageins: 474657.
Pageouts: 77330.
Object cache: 478244 hits of 1592868 lookups (30% hit rate)
Just for comparison, here's the output of "free -p" shortly after that (those of you who haven't investigated free since my last blog post are in for a surprise):
total used free
Mem: 262045 244025 18020
Swap: 262144 171028 91116
I'll start by looking at physical memory, because that's a whole topic in itself which I've spoken about so many times I can cover in a self-contained glob. As an aside though, the pagesize is a compile-time setting, and is 4096 bytes on both architectures unless you go compiling your own kernel. If you do, remember that it's got to be a power-of-2 multiple of the hardware page size.
So, physical memory then. Which of the various pages that we're being told about are actually in physical memory? The active ones must be, and so must the wired ones (a page marked as wired may not be swapped, so by definition they're stored in physical RAM). Actually it turns out that the free and inactive pages are both in RAM too; anything that's been handed out to a pager isn't counted as an available page (but the kernel knows where to ask for that page, if it needs it back). There's no information on page counts outside physical RAM, which was why earlier versions of free(1) couldn't report on the available swap space. It also happens not to make much sense to ask how much swap space is available, because it can change without user interaction. Compare this with Linux: if you swapon(2) a 2GB partition and nothing else, then no matter how much swap space is currently in use, the kernel knows that there's 2GB of swap available. Anyway, that's an aside. Add up the number of free, active, inactive and wired pages, multiply by the page size and you should have a number familiar to you as the total amount of RAM installed in your Mac (or arbitrary other Mach OS box).
Next question, what is the amount of unused RAM? I get asked this frequently and my usual answer is: Heh :-). The free tool just looks at free pages, and top(1) goes for this approach too. Other people will say that it's the number of free and inactive pages. And they're right too, I think. Free pages are absolutely free, and have not been claimed by anything. Inactive pages are candidates for reuse, but do actually contain data. An example of when pages might become inactive is that a task is launched and loads a few dynamic libraries in, then quits. It'd be handy to keep those dylibs around just in case (try launching an app like OmniOutliner, quitting, and relaunching. Even on a poor man's metric like # of dock bounces, it was faster the second time, right?) but if the memory needs to be claimed elsewhere, it can be. So the amount of unused RAM is a nontrivial quantity, but if you mean "how much RAM is free" the answer is the free pages. If you mean "how much more RAM could I activate" then it's that in free or inactive pages. You'll have real performance problems if you've got few free pages and and lots of wired pages, because the wired pages can't go anywhere you've got that much memory less for user-space processes to swap in.
Okey dokey, let's have a look at the paging statistics next then. All of the following values are cumulative; they start at 0 at boot time and monotonically increase with (up)time.
The state of a page is known to the kernel in terms of a few variables:
- Empty
- Wired
- Dirt (i.e. has the page been modified)
- Current access privileges (none, read, or read/write)
- Desired access privileges
There are lots of different mechanisms by which the state as described above can be modified, and each change of state triggers at least one "translation fault". These faults are handled by vm_fault(), so every time that's called the translation fault count goes up.
When a new Mach task is generated, it has an inheritance property which describes whether it starts with its own memory objects or whether it receives copies from its parent (as would happen with a fork(2) call). But rather than generating all those copies when the task is launched, the memory manager gives it shadow objects which refer to the original memory objects. The task doesn't receive its own copy until it tries to write to the shadow object, at which point that generates a copy-on-write fault and the local version of the memory is finally generated.
"Pages zero-filled" describes exactly its own purpose, I hope.
A reactivated page is one that shifts from being inactive to active, as in the example of re-launching an application given above. Reactivating pages means less going out to disk to recover the data, which means faster performance. Finally, a pagein occurs whenever a new page is requested from the pager (basically, whenever a peek or poke into memory that has not yet been dealt with occurs) and a pageout occurs whenever a page has to be handed off to a pager to allow another page to be created or paged back in.
The problem with all of the above information is that it's not presented in a way which I'd find useful :-). I'd prefer my RAM statistics to be presented in more comfortable memory units such as megabytes; this is analogous to the default output of df(1) being the 512-byte block. I don't care about most of the other VM statistics at all, because they're only important if you're debugging the memory manager or a pager. Rather than the number of pageouts since I switched the computer on, I'd like to know the number of pages currently out (i.e. the amount of swap use). There's a sysctl "vm.swapusage" which reports on swap usage, which I've used in free. The best way to get at swap information would be to interrogate the pager(s), not the kernel, and looking at Apple's I don't think there's currently a way to achieve that. But then there are a few things the dynamic_pager would do if I were world emperor that it currently can't; reporting how far off a high-water alert is, cleanly turning off paging to a particular directory, maintaining a single table of all the files currently in use by all the dynamic_pager instances. Anyway, remember what I said earlier about the amount of swap being changable; all we get from vm.swapusage is the amount of filesystem which is currently dedicated to swap.
One thing which can be achieved with the available information, and which vm_stat doesn't report (neither does free) but top(1) does, and which is incredibly useful, is the differential with respect to time. Go on, fire up top. Where there are the pagein and pageout figures, the little number in brackets tells you the increase since the last report. Now a graph of that over time would tell you what effect the system utilisation was having on the memory utilisation. That'd be sweet.
The final line of vm_stat's output is the object cache hit rate, which is the ratio of objects requested vs. those which were in active memory at time of request. Obviously, the higher the ratio, the fewer reactivations and pageins. On a bunch of different Macs I have access to, the hit rate is between 2% and 30%.
BTW, would you like some references? Here's the OSF Mach documentation (the Principles manual contains a good chapter on virtual memory management), Apple's Kernel Programming guide, and the paper describing Mach's virtual memory management by the guys at CMU.
Edit 2006-06-18 09:07 GMT: stopped claiming that free doesn't know about swap space. It does.
Tuesday, June 13, 2006
free as in, well, free(1)
I also found out enough about the Mach virtual memory manager to realise that there are a few things I want to change about the dynamic pager. Without a degree in operating systems design, this might take a while...
Friday, June 02, 2006
NetBSD running on G5
I actually had a FreeBSD 7.0-SNAP up on my G4 tower a few months ago, hoping to help out with the port, but the kernel was so unstable that I couldn't get anything done at all :(. And I'm not as much of a kernel hacker as auserland hacker...
Tuesday, May 23, 2006
Parallels RC and OPENSTEP works!
Eww, although there's a graphics glitch displaying the "please wait until it's safe to turn off your computer" screen.
Update 2006-06-01 11:08 GMT: and so does RC2.
Thursday, May 18, 2006
New c.l.o-c alternative FAQ
Tuesday, May 16, 2006
This so goes on my CV
Amazingly although the disk hadn't been accessed in over a decade (though was stored in a Cool Dry Place^{TM}), everything worked first time. The little jobby with a keyboard and built-in screen on the left is a Cambridge Z88, being used as an emergency VT52 terminal in this case.
Friday, May 12, 2006
Today's broken user interface award...
...goes to printing on OS X Server. You'll notice that you can add or remove printers via Server Admin: however, if you do add one via that route then the system will try and get the printer to emit its PPD at it. If the printer doesn't do this, then it's a "Generic Postscript Printer" and there's no way to change that.
Unless, of course, you log in at the console (Xserves make that so easy and snappy to achieve) and use the same Printer Setup Utility you'd otherwise have used on the client OS. You can then go back into Server Admin to set up whether a queue is quotad, set up sharing and so on.
Great, so far we've used two apps to configure our printer. Let's just add a third (although this does make sense), because you now have to go into workgroup manager to set up the per-user quotas for the print queues. Fine, and while we're here we'll use managed config to define which printers appear in which print panels. Only, we can't. Shared printers can't be added to the MCX print menus...even if they appear in my print menu.
This is so confusing it's almost fun...
Tuesday, May 09, 2006
Complex things possible
Allow me to introduce a decade-old piece of scotch mist.
Wednesday, April 26, 2006
Parallels 2.1 beta 5
If anyone has different experiences with Beta 5, could they comment here?
Tuesday, April 25, 2006
Too much confidence is good for you
Hi Jonathan, I hope you do not mind receiving unsolicited personal mail.I expect precisely nothing to occur...
Congratulations on your appointment as CEO of Sun. Perhaps now would be a good time, in the spirit of openness and interoperability that Sun is famous for, and in extension of the success of the OpenSolaris project, to revisit that Lighthouse Design code in the basement with a view to releasing it under an acceptable open source licence? Would you be able to also open up the source for Sun's OpenStep implementation, as a reference for Cocoa and GNUstep developers to see how it's done? So doing would give Sun lots of 'good faith' points among Cocoa developers on the Mac platform, and further Sun's position at the front of the 21st Century's open source-centric software market.
By the way, Cocoa really is a great application development platform (as I'm sure you know with your OpenStep experience), and the combination of Apple's UNIX workstations with Sun's Solaris UNIX servers, both sporting a coherent and consistent Cocoa-based application stack and development environment would truly be a wonder to behold ;-).
Regards,
Graham.
Friday, April 21, 2006
The repair permissions fiasco
Yesterday I bought some software from the Apple store (the developers and software shall remain nameless, but suffice it to say that it was an expansion pack for a game). Came home in order to install it, that didn't quite work...I'd installed a (n official) patch for the original game, making its version newer than the version in the expansion pack. OK, blow it away, and reinstall.
The installer is a PEF-based Stuffit thing (I didn't realise people still used either PEF or Stuffit), so doesn't know anything about the Authorisation Services API. OK, switch to an admin user, run the installer. Switch back to regular user, and run the game.
Erm, no in-game text. On any of the UI elements. That's a bit strange. Let's check the FAQ on their website; apparently there's a particular file in the game which contains all of the text, if that's corrupted then no text.
It's not corrupted, it's just unreadable by anyone except the user I installed it as. Quick su-chmod-exit, try again. OK, in-game text. Asks for the licence key. Enter it, gets accepted....asks for the licence key. Enter it, check it, gets accepted....asks for the licence key. Couple more times, checking really carefully now that I've entered the correct key (no acc1d3nt4l el1t3ne5s). Definitely doesn't work.
Back to the website FAQ: I find the file where the licence key gets stored. In the app directory. Guess who it's not writable by? That's right, anyone...turns out that savegames get stored there too. So the application dir has to be writable by anyone who might want to play the game. Erm, that kindof sucks.
So I wrote to the company, explaining what I'd found, and got this response:
The fact that the game couldn't write to the nwncdkey.ini file probably means that OS X is having some sort of file permissions issue on your Mac. You might want to boot to your OS X CD, run the disk utility, and run "Repair Disk" and "Repair Disk Permissions" on your hard drive.
Repeat after me: WTF? Their software is installed by a Stuffit installer, not by Installer.app - this means there's no BOM, so there are no known permissions pertaining to that directory as far as the OS is concerned. If they got the permissions wrong on the way in (not unlikely, a number of my user accounts on various systems have custom umasks, and I bet Stuffit doesn't check that) then Disk Utility isn't going to do anything to fix their mistake. For completeness I ran a verify perms check, and indeed nothing came back (well yes something did, in a different area of the filesystem: I had myself chmodded an area of /Developer but last time I checked, Carbon games don't make heavy use of the developer documentation). But the Cult Of Mac is strong, and its idols shall not be slighted, so running repair permissions is obviously the fix in this situation. Perhaps I should zap the PRAM, start with extensions off, rebuild the Desktop and sacrifice me a goat while I'm at it.
I'm going to write "a polite response" to this particular company's support desk, explaining how repair perms works, why perms errors (in the BOM-perms-don't-match-filesystem-perms sense) were not - could not have been - the cause of my issue and requesting that they switch to an installer platform which, well, understands the Mac OS X filesystem. And that they start writing user-specific data like savegames and so on into ~/Library/Application Support/.
Apple Technical Briefing: Scientific computing and the Intel transition
So yesterday was the UKUUG/Apple tech briefing on the Intel transition. Talking to Eric (Albert, Apple Core OS group) and listening to his talk was very interesting, and gave me new and unexpected insights into the whole business of the transition.
Essentially, the fact that [[NeXT|OPEN]Step|Rhapsody] was already ported to Intel platform hardly helped at all. Or maybe that should be "what Apple did to support the Intel port hardly helped at all", though whether they were right or wrong to use it as they did is not the subject of this post. The reasons are manifold, but largely come down to the fact that technology's moved on.
Porting a UNIX - especially Mach, which has been seen on so many architectures already - to a new platform these days is a trivial exercise, with a small team of engineers who know the platform well you could probably get up to single-user mode in a couple of days, and have a multi-user networked platform up and running in a matter of weeks. Apple pretty much went back to the starting blocks on the OS X Intel port, because so much of the existing Intel port was obsolete. Even Darwin/x86 (as was) didn't help here, because although Darwin/x86 and "the Darwin in Intel OS X" target the same architecture, they're different operating systems on different platforms. None of the existing IOKit drivers are for hardware that's contained in an Intel Mac (in fact, OpenStep/Rhapsody didn't use IOKit at all, let alone the fact that no-one wrote an AirPort Extreme driver for them). The processors which OpenStep and Tiger target are completely different - for instance, Tiger doesn't hit the floating point unit at all but instead uses the vector engine's floating point support; this effectively means compiling for a different CPU. Even the bootloader has to be different, to speak with EFI instead of the legacy BIOS.
But that's only part of the problem, it would seem that the higher-level APIs have nothing to gain from older Intel ports either. OK, so OpenStep was designed portable (and largely OS-independent too, but that's another matter) from the ground up, but Cocoa just ain't OpenStep. Or rather, Cocoa ain't just OpenStep. As well as supporting new classes, and the bindings/CoreData stuff, implementations of the OpenStep APIs have been rewritten over the years such that the fact that Yellow Box runs on Intel doesn't mean that Cocoa will. A specific example, and one I particularly liked, which Eric used was that of NSString/CFString, which now (sometimes) uses UTF-16 as its internal data representation. As does whatever type passes for a text representation in the Carbon world. This is nice, except that when Apple first implemented the UTF-16 support, it seems that some of the people forgot to check for endianness, and some other of the people deliberately ignored endianness (i.e. most UTF-16 is bigendian anyway, and we're running bigendian *cough*, let's assume it's all bigendian). Bring that representation over to the Cocoa/Intel API, and interesting and humourous results can occur. So you end up having to reimplement NSString anyway. It actually turns out that Carbon (at least, the Carbon API itself; apps are a different matter) can be easier to deal with than Cocoa, because there you just assume that any data you have control over has historically always been bigendian, and you treat it appropriately.
Anyway, it turned out that the amount of time spent on porting Cocoa and porting Carbon, including all the respective applications, was more-or-less the same. Which I find surprising.
Thursday, April 20, 2006
Comrades in OpenStep
N.B. slides from today's Intel talk coming soon...ish...
Tuesday, April 18, 2006
Wolf! Wolf!
Thursday, April 13, 2006
The age-old problem
That's a bit like the experience at the moment, getting the NE2K driver for NeXTSTEP onto a Parallels-running installation. Never fear, for I have created NE2k driver floppy which can be used in Parallels.
Edit 2006-04-13 12:03 GMT - changed the download URL, once I remembered how small my bandwidth at SDF is.
Tuesday, April 11, 2006
Parallels and OpenStep: success!
Monday, April 10, 2006
More progress on Parallels
Applying Apple's patches was a little more involved - my first approach was to download them on a Mac, create an ISO image (Apple's tools can do this:
hdiutil makehybrid -iso -o OS42patches.iso OS4patches
rather nicely) then put the ISO in the virtual CD-ROM drive. While the tarballs unpacked correctly, the package they contained was corrupt. So I downloaded the patch (OS42MachUserPatch4 being the latest one, containing the SVGA driver and the y2k fixes) to a Solaris box, then used FTP to get it over to the OS4.2 PC (now that we've got network). This was more successful, and the patch could be installed.
So, now we're fully patched, pull out the "Default VGA" driver and put in the VBE (VESA local bus) one, using Configure.app. Let's choose a low resolution for the first boot, just in case; reading Apple's notes, we should be able to check which modes are available and change at boot time anyway. So I've gone for 640x480x8 (i.e. 256 colours) for the first boot... and it hangs here.
This is actually after I typed "VBE Check"=Yes at the boot prompt, and it didn't show a list of modes...so something's not quite right with the VESA driver. I'll leave it overnight in case it decides suddenly to boot, but I think that reverting to Default VGA is, for the moment, the order of the day.
Edit 2006-04-10 17:36 GMT - some history revision, correcting a spelling mistake and therefore pretending I've never heard of visual basic for apps :-)
Saturday, April 08, 2006
About NSMailDelivery and Message.framework
Firstly, the iSendMail target includes Message.framework, which is where NSMailDelivery is defined. Now, some clips from the iSendMail.m file:
if([NSMailDelivery hasDeliveryClassBeenConfigured]==NO)
{
//can't send mail...
NSLog(@"Cannot send mail!\n");
[pool release];
exit(EXIT_FAILURE);
}
Before I even try to send mails out, I check whether NSMailDelivery is going to work. The clincher is straightforward: if Mail.app can send an email, NSMailDelivery can send an email. There's no way to configure e.g. a SMTP server programatically (unless you want to delve into Mail.app's configuration, that is); if you want to do this, consider using PantoMIME which is a more complete message framework. NSMailDelivery is just a convenience class for accessing Mail.app's message sending capabilities, nothing more. Launch Mail, open a new message, put a To address in and hit Cmd-Shift-D. If it goes, then you can use NSMailDelivery.
NSMutableDictionary *headers=[NSMutableDictionary dictionary];
while((argch=getopt(argc,argv,"f:t:s:"))!=-1)
{
switch(argch)
{
case 'f':
[headers setObject:[NSString stringWithCString:optarg] forKey:@"From"];
break;
case 't':
[headers setObject:[NSString stringWithCString:optarg] forKey:@"To"];
break;
case 's':
[headers setObject:[NSString stringWithCString:optarg] forKey:@"Subject"];
break;
case '?':
default:
usage();
}
}
Headers are defined in a dictionary. Obviously a To header is a requirement, you could probably get away without a Subject. If I wanted to use Mail's default email address I wouldn't supply a From header (but I'd use a different delivery method, see below); and if I wanted custom X-Mailer: or whatever, I'd do it here too. Any header which appears here will make it into the resultant mail, and any header not listed here will not be inserted for you.
NSAttributedString *attrMsg;
NSMutableString *msg=[[NSMutableString alloc] init];
//...
attrMsg=[[NSAttributedString alloc] initWithString:msg];
The message must be an attributed string, whether or not you actually use any attributes. But unless you choose a send a MIME message, any attribute you do use will be lost, read on...
mailSent=[NSMailDelivery deliverMessage:attrMsg headers:headers format:NSASCIIMailFormat protocol:nil];
if(mailSent==YES)
{
NSLog(@"Mail sent successfully.\n");
}
else
{
NSLog(@"Mail delivery failed!\n");
}
a few things to note about the delivery line. Firstly, observe that all of these methods we're using are class methods, not instance methods; I never instantiate anything from Message.framework. If I hadn't set a From address I could have used +deliverMessage:subject:to: but then could only send an ASCII message (which is all iSendMail does, anyway) and would have had to accept whatever From address was default in Mail.app. If your message atributed string contains formatting information (or MIME attachments :-)) then use NSMIMEMailFormat. protocol may as well always be nil, as your choices are that (which uses the preferred transport protocol, which is SMTP at the moment on Mac OS X) or NSSMTPDeliveryProtocol, which is also SMTP.
Friday, April 07, 2006
Networking is the biggest current issue, and replacing the default VGA display with something better would be useful, too. But it works :-)
Oh, and images of the driver disks are available from Apple.
Graphics: default VGA. Play it safe :-). We may be able to do something more fancy, but I'd rather check with Parallels before b0rking my install.
Network: ah. The emulated card is a Realtek 8029(AS), and OpenStep doesn't claim to have support for that. Apparently it's compatible with the NE-2000, but OpenStep doesn't have a driver for that either :-(. I need to ask someone with more Intel-version-of-NeXT noodle than I about that...
Sound: we ignored this for the moment.
SCSI: Nope.
So now we're at the 'installing additional packages' stage, I won't post a screenshot until we're into the OS.
Worryingly, the install notes say that we require a SCSI CD-ROM, but Parallels only provides an IDE one. I don't know how that will pan out, I definitely remember one version of NeXTSTEP coming with an IDE CD-ROM driver. Let's see.
OK, so I've got the OpenStep boot floppy up and running (via some failures of NeXTSTEP to provide the necessary device file to dd, and with tremendous thanks to Simon Cozens for lending use of his /dev/fd0), the compatibility BIOS isn't an issue and the installer dutifully loads:
Now to find the drivers disk...