From patbarron at acm.org Mon Mar 2 12:04:26 2020 From: patbarron at acm.org (Pat Barron) Date: Sun, 1 Mar 2020 21:04:26 -0500 (EST) Subject: [TUHS] What was the release history timeline of Ultrix for VAX/MicroVAX? Message-ID: I was just discussing this with someone today, and I realized this information has fallen out of working memory, and I can't find it on secondary storage anywhere... ;-) Does anyone remember the release history and version numbering of Ultrix for the VAX and MicroVAX (which I believe was referred to at various times as Ultrix-32 and Ultrix-32M, until some point when the "-32" thing was dropped, I think?) I was trying to remember what the various released versions of Ultrix were. The person I was discussing with thinks he remembers a version 1.4 and 1.6, before Ultrix 2.0. I honestly don't remember any 1.x version beyond 1.2. Is there a timeline somewhere? --Pat. From arnold at skeeve.com Mon Mar 2 19:05:41 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Mon, 02 Mar 2020 02:05:41 -0700 Subject: [TUHS] Full set of "Computing Systems" for the cost of shipping Message-ID: <202003020905.02295fvm021618@freefriends.org> Hi All. I have a full set (hardcopy) of the USENIX "Computing Systems" journal. It's just sitting on my shelf taking up space, and I'd like to give them to someone who'll give them a good home. So, first person to reply to me directly, who's willing to pay shipping from Israel, gets them. (Payment via paypal, or preferably a US dollar check sent to my US address.) Thanks! Arnold From arnold at skeeve.com Tue Mar 3 01:43:30 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Mon, 02 Mar 2020 08:43:30 -0700 Subject: [TUHS] Full set of "Computing Systems" for the cost of shipping In-Reply-To: <202003020905.02295fvm021618@freefriends.org> References: <202003020905.02295fvm021618@freefriends.org> Message-ID: <202003021543.022FhUOj002872@freefriends.org> Hi --- These have been claimed. Thanks! Arnold arnold at skeeve.com wrote: > Hi All. > > I have a full set (hardcopy) of the USENIX "Computing Systems" journal. > It's just sitting on my shelf taking up space, and I'd like to give them to > someone who'll give them a good home. > > So, first person to reply to me directly, who's willing to pay > shipping from Israel, gets them. (Payment via paypal, or preferably > a US dollar check sent to my US address.) > > Thanks! > > Arnold From pnr at planet.nl Wed Mar 4 00:19:22 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 3 Mar 2020 15:19:22 +0100 Subject: [TUHS] Early Datakit Message-ID: I’m making a little progress with early Datakit & Unix. It would seem that there were various experimental protocols before URP was arrived upon. The first protocol appears to have been designed by Chesson in 1979. One Fraser paper says: ""The first Datakit protocols used a packet structure that was aligned with cell boundaries. Chesson designed a file transfer protocol that transported data in variable length packets, each ending with a trailer.” I think this protocol is being described in his paper "Datakit Software Architecture” (vol. 2, pp. 20.2.1-20.2.5 Proceedings of the IEEE Int. Conf. on Communications, June 1979, Boston) - which unfortunately does not seem to be available in the IEEE online library. Maybe it will surface some day. Next there was a protocol tied to a Datakit terminal interface board that used an 8-slot packet sequence/acknowledge mechanism that carried over into URP. Not much appears to have been published about this. The most interesting experiment seems to have been a project by Luderer, Che and Marshall to build a Datakit-based distributed Unix in 1980/81. Marshall is credited by Fraser as one of the inventors of the URP protocol. The papers are “A virtual switch as the basis for distributed systems” and “A distributed unix system based on a virtual circuit switch” (available online from the ACM library). The first has a good overview description of how early Datakit worked (including interface hardware) and describes the Network Kernel or “NK protocol”. It would seem that URP combines the flow & retransmission control from the terminal interface board with the concept of a single simple receiver algorithm from NK. Unfortunately the paper does not describe the design of the NK transmitter algorithm(s). It confirms that everything ran on V7, but it remains unclear how the Datakit channels were exposed to the user (I currently assume as a cluster of character devices, with the major number identifying the switch connection and the minor number identifying the channel - similar to what later would be used in V8). The second has a description of a modified Unix system, where clients (“S-Unix”) talk to file servers (“F-Unix”), with the file servers mounted into the local file tree of the client. This appears to predate the work of Weinberger for the V8 network file system and its implementation appears different. Questions: - Are Luderer, Che and Marshall still around? (especially Bill Marshall might still recall some more details of the genesis of the URP protocol) - Any recollections about S/F-Unix and how it did or did not influence V8-V10/Plan9? From jon at fourwinds.com Wed Mar 4 04:15:28 2020 From: jon at fourwinds.com (Jon Steinhart) Date: Tue, 03 Mar 2020 10:15:28 -0800 Subject: [TUHS] Command line options and complexity Message-ID: <202003031815.023IFSlD493028@darkstar.fourwinds.com> OK, this should be good for some conversation. A friend sent me this link today: http://danluu.com/cli-complexity/ From athornton at gmail.com Wed Mar 4 04:44:59 2020 From: athornton at gmail.com (Adam Thornton) Date: Tue, 3 Mar 2020 11:44:59 -0700 Subject: [TUHS] Command line options and complexity In-Reply-To: <202003031815.023IFSlD493028@darkstar.fourwinds.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> Message-ID: I've heard people say that there isn't really any alternative to this kind of complexity for command line tools, but people who say that have never really tried the alternative, something like PowerShell. I have plenty of complaints about PowerShell, but passing structured data around and easily being able to operate on structured data without having to hold metadata information in my head so that I can pass the appropriate metadata to the right command line tools at that right places the pipeline isn't among my complaints3 . Somewhat disingenuous. I mean, yes, that's true, but on the other hand it means that you have to keep the "what Powershell commands operate on what structure" in your head instead, since you can no longer assume the pipelines to be a universal interface. Same basic problem as CMS Pipelines. Fantastically powerful, and nowhere near as easy to compose good functionality as "it's just a byte stream." Adam On Tue, Mar 3, 2020 at 11:16 AM Jon Steinhart wrote: > OK, this should be good for some conversation. A friend sent me this > link today: http://danluu.com/cli-complexity/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From coppero1237 at gmail.com Wed Mar 4 14:11:32 2020 From: coppero1237 at gmail.com (Tyler Adams) Date: Wed, 4 Mar 2020 06:11:32 +0200 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> Message-ID: > These go all the way back to v7 unix, where ls has an option to reverse the sort order (which could have been done by passing the output to tac). Good point. Why was this done in v7 unix and why wasn't it thrown out? Tyler On Tue, Mar 3, 2020, 20:45 Adam Thornton wrote: > I've heard people say that there isn't really any alternative to this kind > of complexity for command line tools, but people who say that have never > really tried the alternative, something like PowerShell. I have plenty of > complaints about PowerShell, but passing structured data around and easily > being able to operate on structured data without having to hold metadata > information in my head so that I can pass the appropriate metadata to the > right command line tools at that right places the pipeline isn't among my > complaints3 . > > Somewhat disingenuous. I mean, yes, that's true, but on the other hand it > means that you have to keep the "what Powershell commands operate on what > structure" in your head instead, since you can no longer assume the > pipelines to be a universal interface. > > Same basic problem as CMS Pipelines. Fantastically powerful, and nowhere > near as easy to compose good functionality as "it's just a byte stream." > > Adam > > On Tue, Mar 3, 2020 at 11:16 AM Jon Steinhart wrote: > >> OK, this should be good for some conversation. A friend sent me this >> link today: http://danluu.com/cli-complexity/ >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Wed Mar 4 16:03:55 2020 From: dave at horsfall.org (Dave Horsfall) Date: Wed, 4 Mar 2020 17:03:55 +1100 (EST) Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> Message-ID: On Wed, 4 Mar 2020, Tyler Adams wrote: > > These go all the way back to v7 unix, where ls has an option to > > reverse the sort order (which could have been done by passing the > > output to tac). > > Good point. Why was this done in v7 unix and why wasn't it thrown out? I seem to recall that "sort -r" was in V6, or perhaps that was one of the programs I'd back-ported from V7 (being stuck with 11/40-class boxes). And speaking of "tac" (which I never saw), I couldn't think of a single use for "rev" (although no doubt I'll now get told). Mind you, you get some amusing output with the "man" command because of the way that the underlining works... -- Dave From arnold at skeeve.com Wed Mar 4 16:48:55 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Tue, 03 Mar 2020 23:48:55 -0700 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> Message-ID: <202003040648.0246mteF029358@freefriends.org> Dave Horsfall wrote: > On Wed, 4 Mar 2020, Tyler Adams wrote: > > > > These go all the way back to v7 unix, where ls has an option to > > > reverse the sort order (which could have been done by passing the > > > output to tac). > > > > Good point. Why was this done in v7 unix and why wasn't it thrown out? There was no tac in V7 Unix. It was first posted to USENET, I don't know by who, and picked up by Linux and *BSD. > And speaking of "tac" (which I never saw), I couldn't think of a single > use for "rev" (although no doubt I'll now get told). It's useful for reading Hebrew sent in plain text email :-). Hebrew is read right to left but stored in physical order (left to right) in files. Arnold From beebe at math.utah.edu Thu Mar 5 00:06:53 2020 From: beebe at math.utah.edu (Nelson H. F. Beebe) Date: Wed, 4 Mar 2020 07:06:53 -0700 Subject: [TUHS] Command line options and complexity Message-ID: Arnold Robbins writes: >> There was no tac in V7 Unix. It was first posted to USENET, I don't >> know by who, and picked up by Linux and *BSD. That brought back memories, and to verify them, I checked the tac.c source code in the latest GNU coreutils test release. It says /* Written by Jay Lepreau (lepreau at cs.utah.edu). GNU enhancements by David MacKenzie (djm at gnu.ai.mit.edu). */ So my memory was right that my old friend Jay was the author. Sadly, we lost him in September 2008: see https://www.legacy.com/obituaries/saltlaketribune/obituary.aspx?page=lifestory&pid=117597321 Jay founded the influential Flux group in advanced networking research: http://www.flux.utah.edu/profile/lepreau ------------------------------------------------------------------------------- - Nelson H. F. Beebe Tel: +1 801 581 5254 - - University of Utah FAX: +1 801 581 4148 - - Department of Mathematics, 110 LCB Internet e-mail: beebe at math.utah.edu - - 155 S 1400 E RM 233 beebe at acm.org beebe at computer.org - - Salt Lake City, UT 84112-0090, USA URL: http://www.math.utah.edu/~beebe/ - ------------------------------------------------------------------------------- From jpl.jpl at gmail.com Thu Mar 5 02:17:46 2020 From: jpl.jpl at gmail.com (John P. Linderman) Date: Wed, 4 Mar 2020 11:17:46 -0500 Subject: [TUHS] Command line options and complexity In-Reply-To: References: Message-ID: The "statute of limitations" must have passed long ago, so I confess to having been the author of the original tac (cat in reverse). I was working on a project that wrote log files, but the logs were very "bursty". Minutes might go by without any activity, followed by a burst of logging activity. We often wanted to see *the most recent* burst of activity, so "tail -f" wouldn't do the job. It would show the *next* burst of activity, which might not occur for quite some time. Somebody posted a functional equivalent on some netnews group, but it was *ghastly*. I think it did seeks of -1 characters at a time to accumulate each line. That would have been fast enough to feed our pathetic 1200 baud terminals, but it would have beat the system to death, and that would have been a disservice to other users. My version did reads of 512 bytes on 512-byte boundaries, so it put much less load on the system. I couldn't bear to see something like the netnews version get adopted. The software release process at the Labs was a bureaucratic nightmare, so I "tossed my version over the wall", into the arms of Andy Tanenbaum, as I recall. He made it public, attributed to "an unknown author". I don't know how Rob Pike got ahold of it, but he recognized that mailbox files had the same bursty growth. Unlike our log files, whose contents were acceptably understandable in reverse order, mail messages were hard to read in reverse order, so he proposed making it possible to recognize the headers at the start of each mail message, and put the entire message out in readable order. I think that was a useful option, but the irony of Rob adding an option to "tac" was hard to overlook. The version out there now was rewritten by Jay Lepreau, it seems: /* * tac.c - Print file segments in reverse order * * Original line-only version by unknown author off the net. * Rewritten in 1985 by Jay Lepreau, Univ of Utah, to allocate memory * dynamically, handle string bounded segments (suggested by Rob Pike), * and handle pipes. */ Dynamic buffer allocation rather than relying on the time-honored 512-bytes-is-enough assumption was a positive, as was supporting Rob's suggestion. Handling pipes strikes me as a waste of code, but hey, anything is better than that version I replaced. On Wed, Mar 4, 2020 at 9:15 AM Nelson H. F. Beebe wrote: > Arnold Robbins writes: > > >> There was no tac in V7 Unix. It was first posted to USENET, I don't > >> know by who, and picked up by Linux and *BSD. > > That brought back memories, and to verify them, I checked the tac.c > source code in the latest GNU coreutils test release. It says > > /* Written by Jay Lepreau (lepreau at cs.utah.edu). > GNU enhancements by David MacKenzie (djm at gnu.ai.mit.edu). */ > > So my memory was right that my old friend Jay was the author. Sadly, > we lost him in September 2008: see > > > https://www.legacy.com/obituaries/saltlaketribune/obituary.aspx?page=lifestory&pid=117597321 > > Jay founded the influential Flux group in advanced networking research: > > http://www.flux.utah.edu/profile/lepreau > > > ------------------------------------------------------------------------------- > - Nelson H. F. Beebe Tel: +1 801 581 5254 > - > - University of Utah FAX: +1 801 581 4148 > - > - Department of Mathematics, 110 LCB Internet e-mail: > beebe at math.utah.edu - > - 155 S 1400 E RM 233 beebe at acm.org > beebe at computer.org - > - Salt Lake City, UT 84112-0090, USA URL: > http://www.math.utah.edu/~beebe/ - > > ------------------------------------------------------------------------------- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at bitblocks.com Thu Mar 5 03:25:31 2020 From: bakul at bitblocks.com (Bakul Shah) Date: Wed, 4 Mar 2020 09:25:31 -0800 Subject: [TUHS] Command line options and complexity In-Reply-To: References: Message-ID: I missed knowing about tac till now. I’ve used tail -r since 1982 when Yost pointed out that tail -r|rev was equivalent to a toy recursive C program I had written to reverse a file. He was almost right! rev(){int c=getchar();if(c==EOF)return;rev();putchar(c);} > On Mar 4, 2020, at 8:19 AM, John P. Linderman wrote: > >  > The "statute of limitations" must have passed long ago, so I confess to having been the author of the original tac (cat in reverse). I was working on a project that wrote log files, but the logs were very "bursty". Minutes might go by without any activity, followed by a burst of logging activity. We often wanted to see the most recent burst of activity, so "tail -f" wouldn't do the job. It would show the next burst of activity, which might not occur for quite some time. Somebody posted a functional equivalent on some netnews group, but it was ghastly. I think it did seeks of -1 characters at a time to accumulate each line. That would have been fast enough to feed our pathetic 1200 baud terminals, but it would have beat the system to death, and that would have been a disservice to other users. My version did reads of 512 bytes on 512-byte boundaries, so it put much less load on the system. I couldn't bear to see something like the netnews version > get adopted. The software release process at the Labs was a bureaucratic nightmare, so I "tossed my version over the wall", into the arms of Andy Tanenbaum, as I recall. He made it public, attributed to "an unknown author". > > I don't know how Rob Pike got ahold of it, but he recognized that mailbox files had the same bursty growth. Unlike our log files, whose contents were acceptably understandable in reverse order, mail messages were hard to read in reverse order, so he proposed making it possible to recognize the headers at the start of each mail message, and put the entire message out in readable order. I think that was a useful option, but the irony of Rob adding an option to "tac" was hard to overlook. > > The version out there now was rewritten by Jay Lepreau, it seems: > > /* > * tac.c - Print file segments in reverse order > * > * Original line-only version by unknown author off the net. > * Rewritten in 1985 by Jay Lepreau, Univ of Utah, to allocate memory > * dynamically, handle string bounded segments (suggested by Rob Pike), > * and handle pipes. > */ > > Dynamic buffer allocation rather than relying on the time-honored 512-bytes-is-enough assumption was a positive, as was supporting Rob's suggestion. Handling pipes strikes me as a waste of code, but hey, anything is better than that version I replaced. > >> On Wed, Mar 4, 2020 at 9:15 AM Nelson H. F. Beebe wrote: >> Arnold Robbins writes: >> >> >> There was no tac in V7 Unix. It was first posted to USENET, I don't >> >> know by who, and picked up by Linux and *BSD. >> >> That brought back memories, and to verify them, I checked the tac.c >> source code in the latest GNU coreutils test release. It says >> >> /* Written by Jay Lepreau (lepreau at cs.utah.edu). >> GNU enhancements by David MacKenzie (djm at gnu.ai.mit.edu). */ >> >> So my memory was right that my old friend Jay was the author. Sadly, >> we lost him in September 2008: see >> >> https://www.legacy.com/obituaries/saltlaketribune/obituary.aspx?page=lifestory&pid=117597321 >> >> Jay founded the influential Flux group in advanced networking research: >> >> http://www.flux.utah.edu/profile/lepreau >> >> ------------------------------------------------------------------------------- >> - Nelson H. F. Beebe Tel: +1 801 581 5254 - >> - University of Utah FAX: +1 801 581 4148 - >> - Department of Mathematics, 110 LCB Internet e-mail: beebe at math.utah.edu - >> - 155 S 1400 E RM 233 beebe at acm.org beebe at computer.org - >> - Salt Lake City, UT 84112-0090, USA URL: http://www.math.utah.edu/~beebe/ - >> ------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Thu Mar 5 07:17:08 2020 From: dave at horsfall.org (Dave Horsfall) Date: Thu, 5 Mar 2020 08:17:08 +1100 (EST) Subject: [TUHS] Command line options and complexity In-Reply-To: <202003040648.0246mteF029358@freefriends.org> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <202003040648.0246mteF029358@freefriends.org> Message-ID: On Tue, 3 Mar 2020, arnold at skeeve.com wrote: >> And speaking of "tac" (which I never saw), I couldn't think of a single >> use for "rev" (although no doubt I'll now get told). > > It's useful for reading Hebrew sent in plain text email :-). Hebrew is > read right to left but stored in physical order (left to right) in > files. Ah, of course :-) And Arabic too, as I recall. -- Dave From random832 at fastmail.com Thu Mar 5 07:50:34 2020 From: random832 at fastmail.com (Random832) Date: Wed, 04 Mar 2020 16:50:34 -0500 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> Message-ID: <5019a751-d69a-4839-9a56-b977b275070d@www.fastmail.com> On Tue, Mar 3, 2020, at 13:44, Adam Thornton wrote: > I've heard people say that there isn't really any alternative to this > kind of complexity for command line tools, but people who say that have > never really tried the alternative, something like PowerShell. I have > plenty of complaints about PowerShell, but passing structured data > around and easily being able to operate on structured data without > having to hold metadata information in my head so that I can pass the > appropriate metadata to the right command line tools at that right > places the pipeline isn't among my complaints3 > . > > Somewhat disingenuous. I mean, yes, that's true, but on the other hand > it means that you have to keep the "what Powershell commands operate on > what structure" in your head instead, since you can no longer assume > the pipelines to be a universal interface. Sure, but "stdin is a sequence of any type, and the argument is an expression that operates on that type or the name of a property that that type has" is universal enough. The part that has to operate on a specific structure isn't the command, it's the arguments. For example, a powershell pipeline to produce a list of files sorted by modified date is: gci . | sort lastwritetime | select name all three *commands* are universal - not all objects have a "lastwritetime" and "name" property, but sort and select can operate on any property that the sequence of objects passed into it has. (gci is an alias for get-childitem... it also has aliases ls and dir, but I'm emphasizing that it's not exclusive to directories) *assuming that ls -t didn't exist*, to do this with unix tools that operate on text you would need: ls -l | [somehow convert the date to a sortable format, probably in awk] | sort | [somehow pick the filename alone out of the output - possibly with cut or sed or awk again] and it's very difficult to get tools like awk, sort, and cut to work on formats that contain more than one field that may contain embedded spaces (you can just about get away with it for ls output because the date is always three "words"). A significant portion of ls's options are related to sorting, because you can sort based on fields that are either not present in the output, or are not in a format that can be sorted textually. Maybe it would be enough to have the universal interface be "tables" (i.e. text streams in some format that supports adequate escaping of embedded row and column delimiters)... or maybe even just table rows, and let the user deal with memorizing column numbers (or let each originating command support a fully general way to specify what columns are requested, as ps alone does on modern systems) Of course, this isn't *really* different from allowing any data structure - after all, the value for any field could itself be a fully escaped table in text format. The benefit of having actual data structures with types is that when you *don't* end the pipeline with select, each object knows how to print itself [files print mode, mtime, size, and name in a human-readable format, more or less equivalent to ls -l] rather than just dumping out every single field that you might want sort or select to operate on. From random832 at fastmail.com Thu Mar 5 08:03:20 2020 From: random832 at fastmail.com (Random832) Date: Wed, 04 Mar 2020 17:03:20 -0500 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> Message-ID: <43d31402-fd1a-4324-a8fb-ef72406d41f3@www.fastmail.com> I put a lot of thoughts in my previous message, but hit send before thinking of a good way to summarize my main point... On Tue, Mar 3, 2020, at 13:44, Adam Thornton wrote: > Somewhat disingenuous. I mean, yes, that's true, but on the other hand > it means that you have to keep the "what Powershell commands operate on > what structure" in your head instead, since you can no longer assume > the pipelines to be a universal interface. The thing is, each Unix command imposes an implied structure on its input, so it's not *really* a universal interface. Some operate on lines as free text, some operate on space-delimited fields [with no good way to escape them, though some do support an IFS environment variable to at least change the delimiter], some work best with fixed-width fields. Few provide a way to embed delimiters [be they newline/null for record separator, tab/comma/space field separators, or a user-defined separator for commands that support that] within a value. Sort requires all values to be comparable as either strings or numbers. Most commands you might want to use as a source in a pipeline also expect to be used directly for human-readable output, so they produce output that can be difficult to use for further processing (e.g. dates in ls, which not only can't be sorted directly, but also are limited to minutes for dates in the past year, and days for dates before that, and are in the local time zone) Hardly *any* commands you'd use in a pipeline really operate on unstructured bytes. Compression, I suppose. But other than that, you have just as much need to know what commands operate on what structure in Unix as in Powershell - the only difference is that the serialization is explicitly part of the interface... and due to the typical inability to escape delimiters, leaky. From terry at jon.es Thu Mar 5 09:25:04 2020 From: terry at jon.es (Terry Jones) Date: Thu, 5 Mar 2020 00:25:04 +0100 Subject: [TUHS] Command line options and complexity In-Reply-To: <43d31402-fd1a-4324-a8fb-ef72406d41f3@www.fastmail.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <43d31402-fd1a-4324-a8fb-ef72406d41f3@www.fastmail.com> Message-ID: On Wed, Mar 4, 2020 at 11:04 PM Random832 wrote: > Hardly *any* commands you'd use in a pipeline really operate on > unstructured bytes. Compression, I suppose. But other than that, you have > just as much need to know what commands operate on what structure in Unix > as in Powershell - the only difference is that the serialization is > explicitly part of the interface... and due to the typical inability to > escape delimiters, leaky. > Another difference is that probably most people on this list are extremely familiar with the various quirks and I/O nuances of the tools many have been using every day for decades. Just as the native speakers of a natural language can't so easily see/appreciate its complexity (e.g., pronunciation in English!), I suspect many of us have internalized these idiosyncrasies. I teach occasional shell/Python courses to absolute beginners (no computing experience at all) and came to appreciate how weird the shell is (in the sense of having baked-in historical accidents that cannot / will not / should not be "corrected"). Some of my appreciation of that was due to discussions on this list (e.g., regarding comment syntax, and the : command) - so thanks! I know what follows won't be to everyone's taste, but I like Python and I love shell pipelines, so I tried to write a shell that gave you both and which allowed fairly free mixing of invoking UNIX tools and running Python. You can send anything down its pipelines - lines of text, atoms, numbers, Python objects, whatever (in the Python _ variable). Of course the receiving end of the pipeline needs to know (or figure out) what it's getting. One advantage is that you have a carefully designed programming language (no offence intended!) underlying the shell, so you can e.g., write shell functions in Python (and put them in a start-up file if you want) and just pipe regular UNIX output into them and pipe their output into whatever's next (more Python, another UNIX command, etc). Probably almost no one would actually want to regularly do the following on the command line, but you could: >>> from os import stat >>> def fd(): return [name for (name, time) in sorted((f, stat(f).st_mtime) for f in _)] >>> ls | fd() | tail -n 3 Here I've stuck a simple (DSU - see [1]) Python function in between two UNIX commands and use it to get the most recently modified files. You probably wouldn't want to do this either, but you could: >>> seq 0 9 | list(map(lambda x: 2 ** int(x), _)) | tee /tmp/powers-of-two | sum(map(int, _))1023>>> cat /tmp/powers-of-two1248163264128256512 Of course it also lets you do things you *would* want to do :-) More at https://github.com/terrycojones/daudin Python has fairly nice tools for reading and evaluating Python code, which meant that getting a first version of this implemented took only one evening of playing around. It's pretty simple (and still has plenty of rough edges). Apologies if this seems like self-promotion, but I very much enjoy thinking about things in this thread and about how we work with information. I'm also constantly blown away by how elegant UNIX is and how the core ideas have endured. Pipelines are really wonderful, as "natural" alternative to function composition as a mathematician or programmer would do it (see point #1 at https://github.com/terrycojones/daudin#background--thanks), and I wanted to build a shell that preserved that, while giving you Python. The overview of their history on pages 67-70 of bwk's recent book [2] is very interesting. Terry [1] https://en.wikipedia.org/wiki/Schwartzian_transform [2] https://www.amazon.com/UNIX-History-Memoir-Brian-Kernighan/dp/1695978552 -------------- next part -------------- An HTML attachment was scrubbed... URL: From steffen at sdaoden.eu Thu Mar 5 09:19:36 2020 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Thu, 05 Mar 2020 00:19:36 +0100 Subject: [TUHS] Command line options and complexity In-Reply-To: <5019a751-d69a-4839-9a56-b977b275070d@www.fastmail.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <5019a751-d69a-4839-9a56-b977b275070d@www.fastmail.com> Message-ID: <20200304231936.hyOtO%steffen@sdaoden.eu> Random832 wrote in <5019a751-d69a-4839-9a56-b977b275070d at www.fastmail.com>: |On Tue, Mar 3, 2020, at 13:44, Adam Thornton wrote: |> I've heard people say that there isn't really any alternative to this |> kind of complexity for command line tools, but people who say that have |> never really tried the alternative, something like PowerShell. I have |> plenty of complaints about PowerShell, but passing structured data |> around and easily being able to operate on structured data without |> having to hold metadata information in my head so that I can pass the |> appropriate metadata to the right command line tools at that right |> places the pipeline isn't among my complaints3 |> . |> |> Somewhat disingenuous. I mean, yes, that's true, but on the other hand |> it means that you have to keep the "what Powershell commands operate on |> what structure" in your head instead, since you can no longer assume |> the pipelines to be a universal interface. | |Sure, but "stdin is a sequence of any type, and the argument is an \ |expression that operates on that type or the name of a property that \ |that type has" is universal enough. | |The part that has to operate on a specific structure isn't the command, \ |it's the arguments. | |For example, a powershell pipeline to produce a list of files sorted \ |by modified date is: | |gci . | sort lastwritetime | select name ... |*assuming that ls -t didn't exist*, to do this with unix tools that \ |operate on text you would need: | |ls -l | [somehow convert the date to a sortable format, probably in \ |awk] | sort | [somehow pick the filename alone out of the output - \ |possibly with cut or sed or awk again] | |and it's very difficult to get tools like awk, sort, and cut to work \ |on formats that contain more than one field that may contain embedded \ |spaces (you can just about get away with it for ls output because the \ |date is always three "words"). Yes, that is really bad, except only that a lot of output is pretty portables since a very long time. FreeBSD started using libxo in many base utilities, which can output in structured formats. This includes CSV and even CBOR :), i do not know how the latter integrates in Unix text utilities however. (I think the format string syntax, that a bit originates in QT ??, could have been warped to something better, like the Python ones, plus further extensions, however. But it is an improvement to what the standard formats end up with when reordering etc. comes into place.) --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From lyndon at orthanc.ca Thu Mar 5 10:49:04 2020 From: lyndon at orthanc.ca (Lyndon Nerenberg) Date: Wed, 04 Mar 2020 16:49:04 -0800 Subject: [TUHS] Command line options and complexity In-Reply-To: <202003040648.0246mteF029358@freefriends.org> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <202003040648.0246mteF029358@freefriends.org> Message-ID: > > And speaking of "tac" (which I never saw), I couldn't think of a single > > use for "rev" (although no doubt I'll now get told). It's handy for building rhyming dictionaries: rev < /usr/share/dict/web2 | sort | rev > rhymes --lyndon From robpike at gmail.com Thu Mar 5 10:55:36 2020 From: robpike at gmail.com (Rob Pike) Date: Thu, 5 Mar 2020 11:55:36 +1100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: Message-ID: I have no memory of this, but that doesn't mean it's false. Also in my defense, suggesting an option compared to actually adding the code is a lesser crime. Or is it? Anyway I removed all the options from research cat, including -u. That counts for something. -rob On Thu, Mar 5, 2020 at 3:19 AM John P. Linderman wrote: > > The "statute of limitations" must have passed long ago, so I confess to having been the author of the original tac (cat in reverse). I was working on a project that wrote log files, but the logs were very "bursty". Minutes might go by without any activity, followed by a burst of logging activity. We often wanted to see the most recent burst of activity, so "tail -f" wouldn't do the job. It would show the next burst of activity, which might not occur for quite some time. Somebody posted a functional equivalent on some netnews group, but it was ghastly. I think it did seeks of -1 characters at a time to accumulate each line. That would have been fast enough to feed our pathetic 1200 baud terminals, but it would have beat the system to death, and that would have been a disservice to other users. My version did reads of 512 bytes on 512-byte boundaries, so it put much less load on the system. I couldn't bear to see something like the netnews version > get adopted. The software release process at the Labs was a bureaucratic nightmare, so I "tossed my version over the wall", into the arms of Andy Tanenbaum, as I recall. He made it public, attributed to "an unknown author". > > I don't know how Rob Pike got ahold of it, but he recognized that mailbox files had the same bursty growth. Unlike our log files, whose contents were acceptably understandable in reverse order, mail messages were hard to read in reverse order, so he proposed making it possible to recognize the headers at the start of each mail message, and put the entire message out in readable order. I think that was a useful option, but the irony of Rob adding an option to "tac" was hard to overlook. > > The version out there now was rewritten by Jay Lepreau, it seems: > > /* > * tac.c - Print file segments in reverse order > * > * Original line-only version by unknown author off the net. > * Rewritten in 1985 by Jay Lepreau, Univ of Utah, to allocate memory > * dynamically, handle string bounded segments (suggested by Rob Pike), > * and handle pipes. > */ > > Dynamic buffer allocation rather than relying on the time-honored 512-bytes-is-enough assumption was a positive, as was supporting Rob's suggestion. Handling pipes strikes me as a waste of code, but hey, anything is better than that version I replaced. > > On Wed, Mar 4, 2020 at 9:15 AM Nelson H. F. Beebe wrote: >> >> Arnold Robbins writes: >> >> >> There was no tac in V7 Unix. It was first posted to USENET, I don't >> >> know by who, and picked up by Linux and *BSD. >> >> That brought back memories, and to verify them, I checked the tac.c >> source code in the latest GNU coreutils test release. It says >> >> /* Written by Jay Lepreau (lepreau at cs.utah.edu). >> GNU enhancements by David MacKenzie (djm at gnu.ai.mit.edu). */ >> >> So my memory was right that my old friend Jay was the author. Sadly, >> we lost him in September 2008: see >> >> https://www.legacy.com/obituaries/saltlaketribune/obituary.aspx?page=lifestory&pid=117597321 >> >> Jay founded the influential Flux group in advanced networking research: >> >> http://www.flux.utah.edu/profile/lepreau >> >> ------------------------------------------------------------------------------- >> - Nelson H. F. Beebe Tel: +1 801 581 5254 - >> - University of Utah FAX: +1 801 581 4148 - >> - Department of Mathematics, 110 LCB Internet e-mail: beebe at math.utah.edu - >> - 155 S 1400 E RM 233 beebe at acm.org beebe at computer.org - >> - Salt Lake City, UT 84112-0090, USA URL: http://www.math.utah.edu/~beebe/ - >> ------------------------------------------------------------------------------- From khm at sciops.net Thu Mar 5 12:05:17 2020 From: khm at sciops.net (Kurt H Maier) Date: Wed, 4 Mar 2020 18:05:17 -0800 Subject: [TUHS] Command line options and complexity In-Reply-To: References: Message-ID: <20200305020517.GA13872@wopr> On Wed, Mar 04, 2020 at 11:17:46AM -0500, John P. Linderman wrote: > I think that was a useful option, but the irony of Rob > adding an option to "tac" was hard to overlook. tac came back from Jersey waving flags? khm From ken at google.com Thu Mar 5 14:17:26 2020 From: ken at google.com (Ken Thompson) Date: Wed, 4 Mar 2020 20:17:26 -0800 Subject: [TUHS] Command line options and complexity In-Reply-To: <20200305020517.GA13872@wopr> References: <20200305020517.GA13872@wopr> Message-ID: do i get a prize: ls -tj /bin/ls: illegal option -- j usage: ls [-ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1] [file ...] On Wed, Mar 4, 2020 at 6:06 PM Kurt H Maier wrote: > On Wed, Mar 04, 2020 at 11:17:46AM -0500, John P. Linderman wrote: > > I think that was a useful option, but the irony of Rob > > adding an option to "tac" was hard to overlook. > > tac came back from Jersey waving flags? > > khm > -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug at cs.dartmouth.edu Thu Mar 5 14:57:19 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Wed, 04 Mar 2020 23:57:19 -0500 Subject: [TUHS] Command line options and complexity Message-ID: <202003050457.0254vJ2Y142485@tahoe.cs.Dartmouth.EDU> > These go all the way back to v7 unix, where ls has an option to reverse the sort order (which could have been done by passing the output to tac). A cool idea, but tac was not in v7. And tail didn't get the -r option until v8. As for rev, I don't know why it was first written, but one use was to examine suffixes--a kind of thing that several word lovers in the Unix lab were prone to do. Apropos of using rev to make rhyming dictionaries, Walker's Rhyming Dictionary was published decades before Noah Webster's dictionary appeared and stayed in print for about 200 years. Notionally the relation between webster and walker is rev walker Doug From salewski at att.net Thu Mar 5 16:12:24 2020 From: salewski at att.net (Alan D. Salewski) Date: Thu, 5 Mar 2020 01:12:24 -0500 Subject: [TUHS] Command line options and complexity In-Reply-To: <5019a751-d69a-4839-9a56-b977b275070d@www.fastmail.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <5019a751-d69a-4839-9a56-b977b275070d@www.fastmail.com> Message-ID: <20200305061224.GL24454@att.net> On 2020-03-04 16:50:34, Random832 spake thus: [...] > Sure, but "stdin is a sequence of any type, and the argument is an expression that operates on that type or the name of a property that that type has" is universal enough. > > The part that has to operate on a specific structure isn't the command, it's the arguments. > > For example, a powershell pipeline to produce a list of files sorted by modified date is: > > gci . | sort lastwritetime | select name > > all three *commands* are universal - not all objects have a "lastwritetime" and "name" property, but sort and select can operate on any property that the sequence of objects passed into it has. There are some examples of that type of thing in widely used Unix tools; my use of 'sort -k1,1n' further down is demonstrating such a use case (the 'sort' command is being told that it is operating on numbers). But beyond some lowest common denominator types ("number", "string", ...) how many commands can really usefully operate on a large number of types? For example, a program that can operate on IP addresses is probably doing something different than a program that wants to operate on email addresses. I could see where named properties of some object can be used more generally than types, but again there are widely used tools that do do that (e.g., jq(1)). IMHO, though, they are more cumbersome to use than most of the commands I need to use minute to minute. > (gci is an alias for get-childitem... it also has aliases ls and dir, but I'm emphasizing that it's not exclusive to directories) > > *assuming that ls -t didn't exist*, to do this with unix tools that operate on text you would need: > > ls -l | [somehow convert the date to a sortable format, probably in awk] | sort | [somehow pick the filename alone out of the output - possibly with cut or sed or awk again] (Just nit-picking at this particular example) You could do it without ls[0]: $ stat -c '%Y %n' * | sort -k1,1n | xargs -L1 sh -c 'echo "$@"' That doesn't seem so bad to me, but if it was something I needed regularly I'd of course put it in an alias[1] or (more likely) a short script file. > and it's very difficult to get tools like awk, sort, and cut to work on formats that contain more than one field that may contain embedded spaces (you can just about get away with it for ls output because the date is always three "words"). [...] Yes, that's often true. And when I enounter it I typically start out by seeing if I can inject and remove tokens in the data at key places in the pipeline. Beyond anything trivial, though, I then quickly start reaching for tools to put the data into some form that more easily allow for it (CSV, JSON, ...). But that invariably adds other complications (such as the need to find or build tools to marshal/unmarshal the data, and to deal with data-domain-specific notions of null-vs-empty-string). For the (more common (for me)) case where there is only one field that contains embedded spaces, I just try to get 'em at the end of the line and let the shell deal with it: $ some-command | while read -r first second rest; do ... ; done > Maybe it would be enough to have the universal interface be "tables" (i.e. text streams in some format that supports adequate escaping of embedded row and column delimiters)... or maybe even just table rows, and let the user deal with memorizing column numbers (or let each originating command support a fully general way to specify what columns are requested, as ps alone does on modern systems) Of course, this isn't *really* different from allowing any data structure - after all, the value for any field could itself be a fully escaped table in text format. [...] Well, in some sense with byte streams you have a table of newline-delimited bytes (rows), and byte subfields separated by whitespace (columns). And anything on top of that could (in some context, and with some syntax) be considered just further escaped tables in text format. I think that's essentially the same thing that you said, only with the outermost table syntax removed. But like you said, this isn't really different from allowing any data structure. Importantly, though, it doesn't impose any particular data structure, either. I've worked at a couple of different places that had in-house tools for working with explicit table semantics in command line suites, and where they fit the data domain, that was hugely useful. Generally speaking, they were special purpose enough to warrant their own tools, but still general purpose enough to be composable (were designed for use in shell pipelines) and applicable in domains beyond the intentions of their original authors. Still, the burden of "thinking in tables" would make them too heavyweight for a lot of common use cases. Sometimes my data structure is "paragraphs of text": $ lorem -p 3 | perl -00 -wnle '2 == $. && print' | wc -w Other times I want a tree (JSON, s-expressions, ...), or even a stream of trees[2]. I consider it a feature that these more complex data structures are not assumed or imposed in contexts where they are not needed. Take care, -Al [0] You could get 'ls' to do it, too, (without '-t') but here the use of TIME_STYLE is a presumably non-portable (but handy!) GNU-ism: $ TIME_STYLE='+%s' ls -l | tail -n +2 | sort -k6,6n | xargs -L1 sh -c 'shift 5; echo "$@"' It's different from the '-t' option, though, in that it forces a predicatable date field format in the output of 'ls -l', so side-steps the need for downstream date parsing altogether and simply jumps into sorting (after chopping off the 'total N' header (groans all around)). [1] E.g., $ # read 'bmt' as: "by mtime" $ alias bmt='stat -c "%Y %n" * | sort -k1,1n | xargs -L1 sh -c '"'echo "'"$@"'"'" $ bmt [2] Probably flattened. From crossd at gmail.com Fri Mar 6 00:53:31 2020 From: crossd at gmail.com (Dan Cross) Date: Thu, 5 Mar 2020 09:53:31 -0500 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200305020517.GA13872@wopr> Message-ID: On Wed, Mar 4, 2020 at 11:18 PM Ken Thompson via TUHS wrote: > do i get a prize: > Depends on whether you do your grocery shopping at Trader Joe's. ls -tj > /bin/ls: illegal option -- j > usage: ls [-ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1] [file ...] > Very nice. Wasn't there something in the fortune file at one point about the "Monty Python and the Holy Grail" bridge crossing scene where the question was, "what $n$ lower case letters are not options to ls(1)?" - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Fri Mar 6 06:54:13 2020 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 6 Mar 2020 07:54:13 +1100 (EST) Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <202003040648.0246mteF029358@freefriends.org> Message-ID: On Wed, 4 Mar 2020, Lyndon Nerenberg wrote: (Uses for "rev") > It's handy for building rhyming dictionaries: > > rev < /usr/share/dict/web2 | sort | rev > rhymes Neat! -- Dave From dave at horsfall.org Fri Mar 6 07:50:45 2020 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 6 Mar 2020 08:50:45 +1100 (EST) Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200305020517.GA13872@wopr> Message-ID: On Wed, 4 Mar 2020, Ken Thompson via TUHS wrote: > do i get a prize: > ls -tj > /bin/ls: illegal option -- j > usage: ls [-ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1] [file ...] Another candidate for option-cleansing... Interesting; I get different options with the Mac and FreeBSD: Mac: usage: ls [-ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1] [file ...] FreeBSD: usage: ls [-ABCFGHILPRSTUWZabcdfghiklmnopqrstuwxy1,] [-D format] [file ...] So FreeBSD has added up "y,D:" (in getopt(3)-speak); my eyes are burning... -- Dave From imp at bsdimp.com Fri Mar 6 07:56:58 2020 From: imp at bsdimp.com (Warner Losh) Date: Thu, 5 Mar 2020 14:56:58 -0700 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200305020517.GA13872@wopr> Message-ID: On Thu, Mar 5, 2020 at 2:51 PM Dave Horsfall wrote: > On Wed, 4 Mar 2020, Ken Thompson via TUHS wrote: > > > do i get a prize: > > ls -tj > > /bin/ls: illegal option -- j > > usage: ls [-ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1] [file ...] > > Another candidate for option-cleansing... Interesting; I get different > options with the Mac and FreeBSD: > > Mac: > > usage: ls [-ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1] [file ...] > > FreeBSD: > > usage: ls [-ABCFGHILPRSTUWZabcdfghiklmnopqrstuwxy1,] [-D format] > [file ...] > > So FreeBSD has added up "y,D:" (in getopt(3)-speak); my eyes are burning... > FreeBSD wouldn't need -, if there were a good filter to add , to large numbers... Some of the proliferation of options has been due to a lack of proper building-blocks.... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From ches at cheswick.com Fri Mar 6 08:01:00 2020 From: ches at cheswick.com (William Cheswick) Date: Thu, 5 Mar 2020 17:01:00 -0500 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <202003040648.0246mteF029358@freefriends.org> Message-ID: <375CF120-7AB2-4D62-A295-07DDB30AF5CF@cheswick.com> My use for rev(1): uniq(1)’s -f ignores the first fields of a line. If you want it to ignore the last fields: rev | uniq -f | rev ches From dds at aueb.gr Fri Mar 6 08:17:15 2020 From: dds at aueb.gr (Diomidis Spinellis) Date: Fri, 6 Mar 2020 00:17:15 +0200 Subject: [TUHS] Command line options and complexity In-Reply-To: <202003050457.0254vJ2Y142485@tahoe.cs.Dartmouth.EDU> References: <202003050457.0254vJ2Y142485@tahoe.cs.Dartmouth.EDU> Message-ID: <16a4760f-7bab-c36a-4df7-1df6fc40694d@aueb.gr> On 05-Mar-20 6:57, Doug McIlroy wrote: >> These go all the way back to v7 unix, where ls has an option to reverse > the sort order (which could have been done by passing the output to tac). > > A cool idea, but tac was not in v7. And tail didn't get the -r > option until v8. Tail acquired a -r option between 3BSD [1] and 4BSD [2]. I remember using that option on SunOS in 1990 as part of a prank we played on a friend at the university. On the Sun 3 workstations we were using at the time, one could enter the monitor/debugger program by pressing L1-A. By remotely logging into a workstation and running a shell loop, one could ensure that when the monitor was entered the active program would be that shell. It was then easy to modify the uid field for the active process (the loop-running shell) and set it to zero. After exiting the monitor, a subshell launched from that shell would have full root privileges. All we had to do was wait for the friend to lock his workstation when taking a break in order to obtain root privileges on his workstation and then change to his uid in order to modify his files via NFS on the university's Gould file server. Based on this capability, I wrote the following script that would rename all our friend's files and directories to words from the dictionary. The script also created (via tail -r) another script that would undo this change. #!/bin/sh TMP=/tmp DIR=$1 FILES=$TMP/f.$$ WORDS=$TMP/w.$$ CMD=$TMP/c.$$ REV=$TMP/r.$$ trap '' 0 1 2 3 15 find $DIR -depth -print >$FILES head -`wc -l <$FILES|sed 's/[ ]*//'` /usr/dict/words >$WORDS paste $FILES $WORDS | sed -e ' /^\. /d s/\(.*\)\/\(.*\) \(.*\)/mv \1\/\2 \1\/\3/ ' >$CMD rm $FILES $WORDS tail -r $CMD | sed -e ' s/mv \(.*\) \(.*\)/mv \2 \1/ ' >$REV sh <$CMD rm $CMD Unfortunately, it turned out that tail -r had a limit on the number of lines it could reverse. Although the script and its undo worked fine on a test set of a small number of files, when run on our friend's directory it created a faulty undo script. Our friend ended up graduating with files named "abaca" and "abacinate". [1] https://dspinellis.github.io/manview/?src=https%3A%2F%2Fraw.githubusercontent.com%2Fdspinellis%2Funix-history-repo%2FBSD-3%2Fusr%2Fman%2Fman1%2Ftail.1&name=BSD%203%3A%20tail(1)&link=https%3A%2F%2Fgithub.com%2Fdspinellis%2Funix-history-repo%2Fblob%2FBSD-3%2Fusr%2Fman%2Fman1%2Ftail.1 [2] https://dspinellis.github.io/manview/?src=https%3A%2F%2Fraw.githubusercontent.com%2Fdspinellis%2Funix-history-repo%2FBSD-4%2Fusr%2Fman%2Fman1%2Ftail.1&name=BSD%204%3A%20tail(1)&link=https%3A%2F%2Fgithub.com%2Fdspinellis%2Funix-history-repo%2Fblob%2FBSD-4%2Fusr%2Fman%2Fman1%2Ftail.1 -- Diomidis Spinellis Free edX MOOC on Unix Tools: Data, Software, and Production Engineering https://www.spinellis.gr/unix?tuhs20200306 From pnr at planet.nl Sat Mar 7 06:41:57 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Fri, 6 Mar 2020 21:41:57 +0100 Subject: [TUHS] First appearance of named pipes Message-ID: <8A3F3334-D8C0-4075-B21B-FEAD709C086D@planet.nl> The Luderer paper on distributed Unix has the following paragraph: "A new special UNIX interprocess communication mechanism is the fifo, which provides communication between unrelated processes by associating a new special file type with a file name. Since remote fifos are legal, they can be used for interprocessor communication between S-UNIX machines or between an S-UNIX machine and an F-UNIX machine.” The paper is from late 1981. Maybe I’m especially mud-eyed today, but I cannot see FIFO’s implemented in V7..V8 or 4.1xBSD. When did FIFO’s become a standard Unix feature? Paul From clemc at ccc.com Sat Mar 7 07:06:02 2020 From: clemc at ccc.com (Clem Cole) Date: Fri, 6 Mar 2020 16:06:02 -0500 Subject: [TUHS] First appearance of named pipes In-Reply-To: <8A3F3334-D8C0-4075-B21B-FEAD709C086D@planet.nl> References: <8A3F3334-D8C0-4075-B21B-FEAD709C086D@planet.nl> Message-ID: The first version was from Rand (called "Rand Pipes"). They certainly were available in the mid-70s on Sixth Edition, you have to ask someone like Bruce Borden if they were on Fifth. I think the code is on one of the 'USENIX' tapes in Warren's archives. At this point in time, someone would need to refresh my memory of the details of Rand's implementation compared to what came in the USG systems in the 1980s. For instance, I believe the early versions used mknod(2) to create the "named entity." IIRC early USG did that too, and mkfifo(3) came as part of the POSIX (I have memories of the discussion at a POSIX meeting, but as I say, I've forgotten the details). IIRC there were differences in buffering behavior, flushing, error path between USG's later versions and the original Rand, but I'd have to stare at the code again to remember. On Fri, Mar 6, 2020 at 3:42 PM Paul Ruizendaal wrote: > The Luderer paper on distributed Unix has the following paragraph: > > "A new special UNIX interprocess communication mechanism is the fifo, > which provides communication between unrelated processes by associating a > new special file type with a file name. Since remote fifos are legal, they > can be used for interprocessor communication between S-UNIX machines or > between an S-UNIX machine and an F-UNIX machine.” > > The paper is from late 1981. Maybe I’m especially mud-eyed today, but I > cannot see FIFO’s implemented in V7..V8 or 4.1xBSD. When did FIFO’s become > a standard Unix feature? > > Paul > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Sat Mar 7 07:10:34 2020 From: clemc at ccc.com (Clem Cole) Date: Fri, 6 Mar 2020 16:10:34 -0500 Subject: [TUHS] First appearance of named pipes In-Reply-To: References: <8A3F3334-D8C0-4075-B21B-FEAD709C086D@planet.nl> Message-ID: BTW: My memory is that Joy did not like them for some reason, probably because they were not as sexy as some of the stuff Accent could do (but that's a guess -- I've forgotten). So with 4.2, Joy created Unix domain sockets. BTW: a slow cache refresh is occurring in my brain ... I remember one of the things that there was a lot of arguing/moaning about at the time was the directionality of such a feature. Bruce's hack from the mid-70s was unidirectional and you needed two pipes to go both ways. On Fri, Mar 6, 2020 at 4:06 PM Clem Cole wrote: > The first version was from Rand (called "Rand Pipes"). They > certainly were available in the mid-70s on Sixth Edition, you have to ask > someone like Bruce Borden if they were on Fifth. I think the code is on > one of the 'USENIX' tapes in Warren's archives. > > At this point in time, someone would need to refresh my memory of the > details of Rand's implementation compared to what came in the USG systems > in the 1980s. For instance, I believe the early versions used mknod(2) to > create the "named entity." IIRC early USG did that too, and mkfifo(3) > came as part of the POSIX (I have memories of the discussion at a POSIX > meeting, but as I say, I've forgotten the details). > > IIRC there were differences in buffering behavior, flushing, error path > between USG's later versions and the original Rand, but I'd have to stare > at the code again to remember. > > On Fri, Mar 6, 2020 at 3:42 PM Paul Ruizendaal wrote: > >> The Luderer paper on distributed Unix has the following paragraph: >> >> "A new special UNIX interprocess communication mechanism is the fifo, >> which provides communication between unrelated processes by associating a >> new special file type with a file name. Since remote fifos are legal, they >> can be used for interprocessor communication between S-UNIX machines or >> between an S-UNIX machine and an F-UNIX machine.” >> >> The paper is from late 1981. Maybe I’m especially mud-eyed today, but I >> cannot see FIFO’s implemented in V7..V8 or 4.1xBSD. When did FIFO’s become >> a standard Unix feature? >> >> Paul >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jnc at mercury.lcs.mit.edu Sat Mar 7 08:44:31 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Fri, 6 Mar 2020 17:44:31 -0500 (EST) Subject: [TUHS] First appearance of named pipes Message-ID: <20200306224431.D226C18C080@mercury.lcs.mit.edu> > From: Paul Ruizendaal > The paper is from late 1981. ... When did FIFO's become a > standard Unix feature? Err, V4? :-) At least, that's when pipes arrived (I think - we don't have V4 sources, but there are indications that's when they appeared), and a pipe is a FIFO. RAND ports just allowed (effectively) a pipe to have a name in the file system. The implementation of both is pretty straight-forward. A pipe is just a file which has a maximum length, after which the writer is blocked. A port is just a pipe (it uses the pipe code) whose inode appears in the file system. > From: Clem Cole > I think the code is on one of the 'USENIX' tapes in Warren's archives. Doc is here: https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/doc/ipc and sources for all that are here: https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/dmr https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/ken (port.c is in 'dmr', not 'ken'where it should be). Noel From heinz at osta.com Sat Mar 7 15:08:35 2020 From: heinz at osta.com (Heinz Lycklama) Date: Fri, 6 Mar 2020 21:08:35 -0800 Subject: [TUHS] First appearance of named pipes In-Reply-To: References: <8A3F3334-D8C0-4075-B21B-FEAD709C086D@planet.nl> Message-ID: <35979d2e-b4e2-2de7-f84b-ce03c915447b@osta.com> Clem, you are correct. mknod() was used in the /usr/group standard in 1984. mkfifo() and mkdir() were part of the IEEE Trial Use Standard in 1986, as well as the adopted IEEE Std 1003.1-1988 (POSIX) standard. Heinz On 3/6/2020 1:10 PM, Clem Cole wrote: > BTW:  My memory is that Joy did not like them for some reason, > probably because they were not as sexy as some of the stuff Accent > could do (but that's a guess -- I've forgotten).  So with 4.2, Joy > created Unix domain sockets. > > BTW: a slow cache refresh is occurring in my brain ... I remember one > of the things that there was a lot of arguing/moaning about at the > time was the directionality of such a feature.  Bruce's hack from the > mid-70s was unidirectional and you needed two pipes to go both ways. > > On Fri, Mar 6, 2020 at 4:06 PM Clem Cole > wrote: > > The first version was from Rand (called "Rand Pipes").   They > certainly were available in the mid-70s on Sixth Edition, you have > to ask someone like Bruce Borden if they were on Fifth.  I think > the code is on one of the 'USENIX' tapes in Warren's archives. > > At this point in time, someone would need to refresh my memory of > the details of Rand's implementation compared to what came in the > USG systems in the 1980s.   For instance, I believe the early > versions used mknod(2) to create the "named entity."   IIRC early > USG did that too, and mkfifo(3) came as part of the POSIX (I have > memories of the discussion at a POSIX meeting, but as I say, I've > forgotten the details). > > IIRC there were differences in buffering behavior, flushing, error > path between USG's later versions and the original Rand, but I'd > have to stare at the code again to remember. > > On Fri, Mar 6, 2020 at 3:42 PM Paul Ruizendaal > wrote: > > The Luderer paper on distributed Unix has the following paragraph: > > "A new special UNIX interprocess communication mechanism is > the fifo, which provides communication between unrelated > processes by associating a new special file type with a file > name. Since remote fifos are legal, they can be used for > interprocessor communication between S-UNIX machines or > between an S-UNIX machine and an F-UNIX machine.” > > The paper is from late 1981. Maybe I’m especially mud-eyed > today, but I cannot see FIFO’s implemented in V7..V8 or > 4.1xBSD. When did FIFO’s become a standard Unix feature? > > Paul > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pnr at planet.nl Sat Mar 7 22:17:09 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sat, 7 Mar 2020 13:17:09 +0100 Subject: [TUHS] First appearance of named pipes In-Reply-To: <20200306224431.D226C18C080@mercury.lcs.mit.edu> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> Message-ID: <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> Thanks for the feedback, all. Rand ports were done in 1977 by Sunshine/Zucker. I’ve only come across Rand Ports in the context of V6 and the Arpa crowd 1977-1981. I’ve never seen a reference to Rand Ports on V7 or later. This of course does not mean that it did not exist. I’ve dug further, and it would seem that named pipes under the name ‘fifo’ appeared first in SysIII (1980). That matches with Luderer’s remark. It does not seem to exist in the Research editions. It only appears in BSD in the Reno release, 1990. All in all, it would seem that ‘fifo’s were a SysV thing for most of the 80’s, with the BSD lineage using domain sockets instead (as Clem mentioned). Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication. Paul PS really nobody on the list recalls Luderer's (et al.) distributed Unix and how it related to other work ?? > On 6 Mar 2020, at 23:44, Noel Chiappa wrote: > > >> From: Paul Ruizendaal > >> The paper is from late 1981. ... When did FIFO's become a >> standard Unix feature? > > Err, V4? :-) At least, that's when pipes arrived (I think - we don't have V4 > sources, but there are indications that's when they appeared), and a pipe is a > FIFO. RAND ports just allowed (effectively) a pipe to have a name in the file > system. > > The implementation of both is pretty straight-forward. A pipe is just a file > which has a maximum length, after which the writer is blocked. A port is > just a pipe (it uses the pipe code) whose inode appears in the file system. > >> From: Clem Cole > >> I think the code is on one of the 'USENIX' tapes in Warren's archives. > > Doc is here: > > https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/doc/ipc > > and sources for all that are here: > > https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/dmr > https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/ken > > (port.c is in 'dmr', not 'ken'where it should be). > > Noel From clemc at ccc.com Sat Mar 7 23:29:05 2020 From: clemc at ccc.com (Clem Cole) Date: Sat, 7 Mar 2020 08:29:05 -0500 Subject: [TUHS] First appearance of named pipes In-Reply-To: <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> Message-ID: below.. in-line On Sat, Mar 7, 2020 at 7:18 AM Paul Ruizendaal wrote: > Thanks for the feedback, all. > > Rand ports were done in 1977 by Sunshine/Zucker. I’ve only come across > Rand Ports in the context of V6 and the Arpa crowd 1977-1981. I’ve never > seen a reference to Rand Ports on V7 or later. This of course does not mean > that it did not exist. > Steve Glaser hacked was playing with them at Tektronix in 1979, as he had put them into their V6 system before I got there IIRC. I switched the user code to use Chesson's MPX in V7, which is why I think they never were used much in V7. I've forgotten what Bruce used for UNET - I'm CC'ing him here, hoping to jog his memory., > > I’ve dug further, and it would seem that named pipes under the name ‘fifo’ > appeared first in SysIII (1980). That matches with Luderer’s remark. It > does not seem to exist in the Research editions. It only appears in BSD in > the Reno release, 1990. All in all, it would seem that ‘fifo’s were a SysV > thing for most of the 80’s, with the BSD lineage using domain sockets > instead (as Clem mentioned). > Yes, that's right. And if you were someone like Masscomp or Pyramid trying to thread both systems, we had both in our kernels. > > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one > of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for > interproces communication. > > Paul > > PS really nobody on the list recalls Luderer's (et al.) distributed Unix > and how it related to other work ?? > > > > On 6 Mar 2020, at 23:44, Noel Chiappa wrote: > > > > > >> From: Paul Ruizendaal > > > >> The paper is from late 1981. ... When did FIFO's become a > >> standard Unix feature? > > > > Err, V4? :-) At least, that's when pipes arrived (I think - we don't > have V4 > > sources, but there are indications that's when they appeared), and a > pipe is a > > FIFO. RAND ports just allowed (effectively) a pipe to have a name in the > file > > system. > > > > The implementation of both is pretty straight-forward. A pipe is just a > file > > which has a maximum length, after which the writer is blocked. A port is > > just a pipe (it uses the pipe code) whose inode appears in the file > system. > > > >> From: Clem Cole > > > >> I think the code is on one of the 'USENIX' tapes in Warren's archives. > > > > Doc is here: > > > > https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/doc/ipc > > > > and sources for all that are here: > > > > https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/dmr > > https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/ken > > > > (port.c is in 'dmr', not 'ken'where it should be). > > > > Noel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dfawcus+lists-tuhs at employees.org Sun Mar 8 02:39:35 2020 From: dfawcus+lists-tuhs at employees.org (Derek Fawcus) Date: Sat, 7 Mar 2020 16:39:35 +0000 Subject: [TUHS] First appearance of named pipes In-Reply-To: <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> Message-ID: <20200307163935.GA57521@clarinet.employees.org> On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote: > > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication. Could that simply be bleed over of terminology from the ARPAnet / Internet usage, in that "socket" is used to refer to protocol end points? i.e. see these from 1970: https://tools.ietf.org/html/rfc54 https://tools.ietf.org/html/rfc55 https://tools.ietf.org/html/rfc60 DF From robpike at gmail.com Sun Mar 8 12:36:14 2020 From: robpike at gmail.com (Rob Pike) Date: Sun, 8 Mar 2020 13:36:14 +1100 Subject: [TUHS] First appearance of named pipes In-Reply-To: <20200307163935.GA57521@clarinet.employees.org> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> Message-ID: Always bemused me that to get a named local I/O connection one ended up with "Unix domain (what does that even mean?) sockets" rather than named pipes, especially since sockets are about as natural a Unix concept as lawn mowers. I've been told, but haven't confirmed, that early sockets didn't even support read and write. They still don't support open and close, and never will. Networks are not intrinsically more special than any other I/O peripheral, but they have become gilded unicorns mounted on rotating hovercrafts compared to the I/O devices Unix supported before them. -rob On Sun, Mar 8, 2020 at 3:48 AM Derek Fawcus wrote: > > On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote: > > > > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication. > > Could that simply be bleed over of terminology from the ARPAnet / Internet > usage, in that "socket" is used to refer to protocol end points? > > i.e. see these from 1970: > > https://tools.ietf.org/html/rfc54 > https://tools.ietf.org/html/rfc55 > https://tools.ietf.org/html/rfc60 > > DF From lm at mcvoy.com Sun Mar 8 12:47:00 2020 From: lm at mcvoy.com (Larry McVoy) Date: Sat, 7 Mar 2020 18:47:00 -0800 Subject: [TUHS] First appearance of named pipes In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> Message-ID: <20200308024700.GE8730@mcvoy.com> I hear you but sockets are here to stay. Sun tried to get rid of them by going to a STREAMS networking stack (not saying that was in any way a better answer, just different). Didn't work, they had to put sockets back, there was just way too much software written around the socket API. I tried to make a more sane interface and never got to something that handled all the edge cases. Did Plan 9 make it sane? If so, care to say how or point me at something like Masscomp's introduction to network programming? On Sun, Mar 08, 2020 at 01:36:14PM +1100, Rob Pike wrote: > Always bemused me that to get a named local I/O connection one ended > up with "Unix domain (what does that even mean?) sockets" rather than > named pipes, especially since sockets are about as natural a Unix > concept as lawn mowers. I've been told, but haven't confirmed, that > early sockets didn't even support read and write. They still don't > support open and close, and never will. > > Networks are not intrinsically more special than any other I/O > peripheral, but they have become gilded unicorns mounted on rotating > hovercrafts compared to the I/O devices Unix supported before them. > > -rob > > On Sun, Mar 8, 2020 at 3:48 AM Derek Fawcus > wrote: > > > > On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote: > > > > > > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ???sockets??? (in today???s parlance) for interproces communication. > > > > Could that simply be bleed over of terminology from the ARPAnet / Internet > > usage, in that "socket" is used to refer to protocol end points? > > > > i.e. see these from 1970: > > > > https://tools.ietf.org/html/rfc54 > > https://tools.ietf.org/html/rfc55 > > https://tools.ietf.org/html/rfc60 > > > > DF -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm From dave at horsfall.org Sun Mar 8 13:06:50 2020 From: dave at horsfall.org (Dave Horsfall) Date: Sun, 8 Mar 2020 14:06:50 +1100 (EST) Subject: [TUHS] First appearance of named pipes In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> Message-ID: On Sun, 8 Mar 2020, Rob Pike wrote: > Always bemused me that to get a named local I/O connection one ended up > with "Unix domain (what does that even mean?) sockets" rather than named > pipes, especially since sockets are about as natural a Unix concept as > lawn mowers. Indeed... > I've been told, but haven't confirmed, that early sockets didn't even > support read and write. They had their own I/O calls such as send()/recv() (and still do). > They still don't support open and close, and never will. Huh; imagine my surprise when I named a function "shutdown()" because it was called at SIGTERM to clean up... > Networks are not intrinsically more special than any other I/O > peripheral, but they have become gilded unicorns mounted on rotating > hovercrafts compared to the I/O devices Unix supported before them. And that's being polite... They are the worst interface that I have ever seen (including OS/360). At a previous $ORKPLACE there was a library that said simply "I am server on port N" and "I want to contact a service on a.b.c.d on port N"; I wish I'd stole^Wborrowed it when we eventually parted company. -- Dave From grog at lemis.com Sun Mar 8 15:26:33 2020 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Sun, 8 Mar 2020 16:26:33 +1100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200305020517.GA13872@wopr> Message-ID: <20200308052632.GD20478@eureka.lemis.com> On Thursday, 5 March 2020 at 14:56:58 -0700, Warner Losh wrote: > On Thu, Mar 5, 2020 at 2:51 PM Dave Horsfall wrote: >> On Wed, 4 Mar 2020, Ken Thompson via TUHS wrote: >> >>> do i get a prize: >>> ls -tj >>> /bin/ls: illegal option -- j >>> usage: ls [-ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1] [file ...] >> >> Another candidate for option-cleansing... Interesting; I get different >> options with the Mac and FreeBSD: >> >> Mac: >> >> usage: ls [-ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1] [file ...] >> >> FreeBSD: >> >> usage: ls [-ABCFGHILPRSTUWZabcdfghiklmnopqrstuwxy1,] [-D format] >> [file ...] >> >> So FreeBSD has added up "y,D:" (in getopt(3)-speak); my eyes are burning... > > FreeBSD wouldn't need -, if there were a good filter to add , to large > numbers... Some of the proliferation of options has been due to a lack of > proper building-blocks.... I wasn't going to join this discussion, but as the perpetrator of all three of the options that Dave complains about, I think it's worth explaining the rationale. First: yes, filters are good. They make for an extraordinarily flexible system. And many options are just bloat. But on the other hand, let's follow on with your example and assume a clever filter, say commafy, which would insert commas as needed in its input: $ ls -l | commafy 5 You really need the 5 (column number), because you can't rely on all large numeric values to require commas. Consider: $ ls -l 939585975893478543543 -rw-r--r-- 2 grog home 1719298048 8 Mar 14:14 939585975893478543543 The alternative would be to have the column number explicitly stated in the filter, but that would make the filter more specific to ls. But do you really want to add that much input when typing interactively into a shell? How much easier it is just to write: $ ls -l, 939585975893478543543 -rw-r--r-- 2 grog home 1,719,298,048 8 Mar 14:14 939585975893478543543 And then there are things that a filter can't easily do, the rationales for -y and -D format. -y is really a workaround for a bug in the POSIX specification for ls(1). From https://pubs.opengroup.org/onlinepubs/009695399/utilities/ls.html: -t Sort with the primary key being time modified (most recently modified first) and the secondary key being filename in the collating sequence. It's not immediately obvious, but these two keys sort in the opposite order. The file name is sorted alphabetically, but the modification time is the other way round (*reverse* chronological). This problem bites you, for example, when you list files from two different cameras that can take more than one image with the same time stamp. FAT timestamps have a granularity of 1 second, so they all end up with exactly the same time stamp. From a diary entry for 24 January 2009 (http://www.lemis.com/grog/diary-jan2009.php?subtitle=%E2%80%9CNot%20a%20bug,%20a%20feature%E2%80%9D:%20episode%204714&article=lsorder#lsorder): === grog at dereel (/dev/ttyp2) ~/Photos/20061223/orig 63 -> ls -lTrt -rwxrwxrwx 1 grog home 2478324 Dec 23 15:35:08 2006 DSCN1325.JPG -rwxr-xr-x 1 grog home 1628592 Dec 23 17:11:00 2006 img_5504.jpg -rwxr-xr-x 1 grog home 1621982 Dec 23 17:11:00 2006 img_5503.jpg -rwxrwxrwx 1 grog home 2583242 Dec 23 17:27:30 2006 DSCN1326.JPG -rwxrwxrwx 1 grog home 2476707 Dec 23 17:27:48 2006 DSCN1327.JPG The file names for images with different timestamps are sorted alphabetically. The file names for images with the same timestamps are sorted in reverse alphabetical order. What to do? Potentially you could write a filter here too, though it wouldn't be simple, because the timestamp representation depends on the age of the file. And you can't just fix the bug, because it has been elevated to a feature. So -y does the right thing. And that date. There are three relatively arbitrary formats, two of them depending on how long ago the timestamp was: -rw-r--r-- 2 grog home 1,719,298,048 8 Mar 14:14 939585975893478543543 -rw-r--r-- 1 grog home 0 24 Sep 2012 foo You can fix that (on FreeBSD and probably on macOS) with the equally unsupported -T flag ("full timestamp"): $ ls -lT 939585975893478543543 foo -rw-r--r-- 2 grog home 1719298048 8 Mar 14:14:58 2020 939585975893478543543 -rw-r--r-- 1 grog home 0 24 Sep 14:42:57 2012 foo Do we need another format? Maybe. Certainly it would help to have a different format if you want to pass the output to a filter that looks at the timestamp. What should it be? Your guess is as good as mine, but probably different. Obvious choices are raw time_t and YYYYMMDDhhmmss. So I introduced the -D option to allow the user to choose his own output format. Is this a good idea? I certainly had pangs of conscience every time, and a non-standard option runs the risk of being incompatible with other systems. For example, Linux uses -T to define the tab size (arguably a better choice for a filter) and -D to produce output for Emacs dired mode. In summary: there's a tradeoff between the elegance of filters and the effort that they require. Adding options has its disadvantages too. You need to remember them, and they can easily become incompatible. But these specific features make life considerably easier and add very little to the size of the executable. I'd be interested to hear of alternative solutions to the issues. Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From jon at fourwinds.com Sun Mar 8 15:32:38 2020 From: jon at fourwinds.com (Jon Steinhart) Date: Sat, 07 Mar 2020 21:32:38 -0800 Subject: [TUHS] Command line options and complexity In-Reply-To: <20200308052632.GD20478@eureka.lemis.com> References: <20200305020517.GA13872@wopr> <20200308052632.GD20478@eureka.lemis.com> Message-ID: <202003080532.0285WcWn1544496@darkstar.fourwinds.com> After following this discussion, I guess that I have a simplistic way to determine whether something should be a dash option or a filter. In general, I'd make a filter if whatever it was doing was applicable to more than one command, a dash option otherwise. Jon From arnold at skeeve.com Sun Mar 8 17:16:41 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sun, 08 Mar 2020 00:16:41 -0700 Subject: [TUHS] First appearance of named pipes In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> Message-ID: <202003080716.0287Gfa0029261@freefriends.org> Rob Pike wrote: > Networks are not intrinsically more special than any other I/O > peripheral, but they have become gilded unicorns mounted on rotating > hovercrafts compared to the I/O devices Unix supported before them. And another one for the fortune file ... :-) Arnold From coppero1237 at gmail.com Sun Mar 8 19:30:57 2020 From: coppero1237 at gmail.com (Tyler Adams) Date: Sun, 8 Mar 2020 11:30:57 +0200 Subject: [TUHS] Command line options and complexity In-Reply-To: <202003080532.0285WcWn1544496@darkstar.fourwinds.com> References: <20200305020517.GA13872@wopr> <20200308052632.GD20478@eureka.lemis.com> <202003080532.0285WcWn1544496@darkstar.fourwinds.com> Message-ID: The idea of a simple rule is great, but the suggested rule fails on sort -u which afaik came after sort | uniq for performance reasons. Another idea on the same vein is that a flag should be added only when the job can be done inside the program and not with stdin/stdout (or no flag can be added if one can reproduce the same behavior using pipelines). So, you need sort -u because only within sort can you get the performance needed to get the job done. But you don't need -h in ls -lh. All the information to render a human readable number is present on stdout of ls -l. You could easily have a filter which renders numbers with options like adding commas, dots, scientific notation, precision, money, units, etc. Tyler On Sun, Mar 8, 2020, 07:33 Jon Steinhart wrote: > After following this discussion, I guess that I have a simplistic way to > determine whether something should be a dash option or a filter. In > general, I'd make a filter if whatever it was doing was applicable to > more than one command, a dash option otherwise. > > Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael at kjorling.se Sun Mar 8 19:51:19 2020 From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=) Date: Sun, 8 Mar 2020 09:51:19 +0000 Subject: [TUHS] Command line options and complexity In-Reply-To: <20200308052632.GD20478@eureka.lemis.com> References: <20200305020517.GA13872@wopr> <20200308052632.GD20478@eureka.lemis.com> Message-ID: On 8 Mar 2020 16:26 +1100, from grog at lemis.com (Greg 'groggy' Lehey): > FAT timestamps have a granularity of 1 second, Not quite. Last modified time is recorded to within two seconds (FAT squeezes the seconds into a 5-bit field, which allows packing a time into two bytes). Other times are recorded with different granularity, sometimes depending on the OS/version used to make the change to the file system. And of course FAT has no concept of time zones; everything is local time, all the time. https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system#Directory_entry has some of the gory details. -- Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se “Remember when, on the Internet, nobody cared that you were a dog?” From ralph at inputplus.co.uk Sun Mar 8 23:07:42 2020 From: ralph at inputplus.co.uk (Ralph Corderoy) Date: Sun, 08 Mar 2020 13:07:42 +0000 Subject: [TUHS] First appearance of named pipes In-Reply-To: <20200308024700.GE8730@mcvoy.com> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308024700.GE8730@mcvoy.com> Message-ID: <20200308130742.0AE102212F@orac.inputplus.co.uk> Hi Larry, > Did Plan 9 make it sane? If so, care to say how or point me at something > like Masscomp's introduction to network programming? There's Plan 9's dial(2) and friends. http://man.cat-v.org/plan_9/2/dial The ctl file of a connection takes commands like ‘hangup’. http://man.cat-v.org/plan_9/3/ip The idea of dial() just doing the right thing lives on with Go's https://golang.org/pkg/net/#pkg-overview -- Cheers, Ralph. From arnold at skeeve.com Sun Mar 8 23:25:16 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sun, 08 Mar 2020 07:25:16 -0600 Subject: [TUHS] First appearance of named pipes In-Reply-To: <20200308130742.0AE102212F@orac.inputplus.co.uk> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308024700.GE8730@mcvoy.com> <20200308130742.0AE102212F@orac.inputplus.co.uk> Message-ID: <202003081325.028DPGEg009475@freefriends.org> Ralph Corderoy wrote: > Hi Larry, > > > Did Plan 9 make it sane? If so, care to say how or point me at something > > like Masscomp's introduction to network programming? > > There's Plan 9's dial(2) and friends. > http://man.cat-v.org/plan_9/2/dial > The ctl file of a connection takes commands like ‘hangup’. > http://man.cat-v.org/plan_9/3/ip > > The idea of dial() just doing the right thing lives on with Go's > https://golang.org/pkg/net/#pkg-overview > > -- > Cheers, Ralph. But more than that is that all files are served via the 9P protocol, even local ones. This is a great intro to Plan 9: https://www.youtube.com/watch?v=6m3GuoaxRNM Just over an hour and a quarter, but well worth the time. Arnold From dfawcus+lists-tuhs at employees.org Mon Mar 9 01:13:32 2020 From: dfawcus+lists-tuhs at employees.org (Derek Fawcus) Date: Sun, 8 Mar 2020 16:13:32 +0100 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> Message-ID: <20200308151332.GA70333@clarinet.employees.org> On Sun, Mar 08, 2020 at 01:36:14PM +1100, Rob Pike wrote: > Always bemused me that to get a named local I/O connection one ended > up with "Unix domain (what does that even mean?) sockets" rather than > named pipes, Yeah - I always found that a bit weird, having to use socketpair() to get a bidirectional "pipe". > I've been told, but haven't confirmed, that > early sockets didn't even support read and write. They still don't > support open and close, and never will. Err - granted on the open; but if my memory serves, close() has been supported on them ever since I started using them ('87). Otherwise the normal fork/dup/close/exec pattern for child processes would not work. Now what would have been useful is a way to have distinct fd's for the local read and write end of (e.g.) a TCP socket - such that one direction could be closed w/o closing the other. Or maybe some fcntl() to dup the bidirectional fd in to a pair of unidirectional fds. That way one could dispense with shutdown for closing one direction, making children and fd passed programs socket agnostic. DF From pnr at planet.nl Mon Mar 9 07:42:16 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sun, 8 Mar 2020 22:42:16 +0100 Subject: [TUHS] First appearance of named pipes Message-ID: <868FEF19-8F8A-4DFA-8773-1C67FB32CE74@planet.nl> > Always bemused me that to get a named local I/O connection one ended up with "Unix domain (what does that even mean?) sockets" rather than named pipes, especially since sockets are about as natural a Unix concept as lawn mowers. I've been told, but haven't confirmed, that early sockets didn't even support read and write. They still don't support open and close, and never will. My interest in Unix networking 1975-1985 originally came from wondering how we came up with this alien feeling socket API as the dominant model. The original ideas for this API are in the recently found CSRG tech reports #4 and #3 - which I hope to discuss on this list in Spring. I think we have to distinguish the API and the underlying paradigms. When it comes to the “Arpa” lineage of Unix networking, the original API model was fully within the open-read-write-close framework. See for instance RFC681 and this document: https://minnie.tuhs.org/cgi-bin/utree.pl?file=BBN-Vax-TCP/doc/net.5.P; the entire BBN network API model fits on a few pages of ‘man’ text. In 1975 Arpa Unix, the network name space was integrated with the file name space, by creating a character special file for each network host. This was possible because at that time an Arpa network address was 8 bits, and this fitted in the minor number; when Arpa addresses were expanded to 24 bits soon after this approach was abandoned (but one could think of a mechanism akin to symbolic links that could have continued the practice). One could have an entry for the local host, e.g. “/n/local” or something like that. In my mind, “socket” does not only refer to the sockets API, but also to the concept of a bi-directional, possibly remote, named pipe; ‘named' as in “discoverable by a possibly unrelated process”, i.e. the in file name space, the network name space if different, etc. [aside: I realise (now) that this is a confusing use of the word socket, but I don’t have a better phrase at hand.] In my opinion, it is this concept that has proven strong and durable, much more so than the socket API itself. When viewed in this definition a ‘fifo’ is a limited form of socket: it is unidirectional, local only (although in the 1981 S/F-Unix it wasn’t) and a server process cannot easily distinguish or delegate individual client connections. The Rand Port was better in the sense that it prefixed each client’s data with a header block. > Networks are not intrinsically more special than any other I/O peripheral, but they have become gilded unicorns mounted on rotating hovercrafts compared to the I/O devices Unix supported before them. -rob "Networks are not intrinsically more special than any other I/O peripheral”: that indeed is the paradigm that underlies Spider-Datakit-streams-STREAMS-Plan9, networks are just an I/O peripheral. There is nothing wrong with that paradigm, excellent systems can be built on top of it. The other paradigm is that the network is a (mostly hidden) substrate that carries bidirectional pipes between processes. It would seem to me that there is nothing wrong with that paradigm either and it can be implemented in a “natural Unix” way as well. > On Sun, Mar 8, 2020 at 3:48 AM Derek Fawcus wrote: > >> > On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote: >> > > >> > > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication. >> > >> > Could that simply be bleed over of terminology from the ARPAnet / Internet >> > usage, in that "socket" is used to refer to protocol end points? I meant ’socket’ in the sense that I described above. “Socket” must be one of the most overloaded words in networking. My understanding is that on the Arpanet the “socket number” was what we now would call a "port number”, although I think it was initially meant to identify a user on a host, rather than a service on a host. In the 1980 BBN TCP implementation “socket” is used to mean “ip address”. A year later, Bill Joy uses “socket” as an API call name. >> > >> > i.e. see these from 1970: >> > >> > https://tools.ietf.org/html/rfc54 >> > https://tools.ietf.org/html/rfc55 >> > https://tools.ietf.org/html/rfc60 >> > >> > DF From jon at fourwinds.com Mon Mar 9 08:04:27 2020 From: jon at fourwinds.com (Jon Steinhart) Date: Sun, 08 Mar 2020 15:04:27 -0700 Subject: [TUHS] First appearance of named pipes In-Reply-To: <868FEF19-8F8A-4DFA-8773-1C67FB32CE74@planet.nl> References: <868FEF19-8F8A-4DFA-8773-1C67FB32CE74@planet.nl> Message-ID: <202003082204.028M4RZX042105@darkstar.fourwinds.com> Rob Pike writes: > Always bemused me that to get a named local I/O connection one ended up with "Unix domain (what does that even mean?) sockets" > rather than named pipes, especially since sockets are about as natural a Unix concept as lawn mowers. I love my Unix lawn mower. The great thing about it is that I don't need a separate lawn mower for each type of grass, and that I can hook things like mulchers up using a pipe. Jon From norman at oclsc.org Mon Mar 9 09:09:01 2020 From: norman at oclsc.org (Norman Wilson) Date: Sun, 08 Mar 2020 19:09:01 -0400 Subject: [TUHS] sockets (was Re: First appearance of named pipes) Message-ID: <1583708946.8301.for-standards-violators@oclsc.org> Derek Fawcus: Yeah - I always found that a bit weird, having to use socketpair() to get a bidirectional "pipe". In the Research system, pipes became bidirectional when they became streams. That happened slightly before my time, but so far as I know it broke absolutely nothing. Some time in the late 1980s, the System V people wanted to allow pipes to be streams, but were worried about the bidirectionality. They proposed to have a new system call to make a bidirectional pipe. I attended a meeting with the relevant programmers and program manager to find out why they thought pipes couldn't just be bi-directional, as they had been without fuss in the Research system for some years. They agreed with me that that was how it ought to be; the trouble was that System V releases all had to pass an official System V Verification Suite (reasonable enough), and that suite checked not only that you could read the one pipe file descriptor and write the other, but that you couldn't do it the wrong way. Wait a minute, I said. I'm pretty sure that's not how the official System V Interface Description reads. Anyone got a current copy handy? We found one, and we looked, and sure enough, the official verification suite was wrong. The specification said that data written to fd[1] must be readable from fd[0], but nothing about the other direction: full-duplex pipes were not required but neither were they outlawed! The programming group was delighted: I'd given them the ammo they needed to do it right (make pipes streams, and make them full-duplex by default). I believe that is how it came out, though the only reference I have is Solaris 10, where the manual page specifically says that what pipe(2) makes is full-duplex (and a stream). I wish POSIX and Linux and the BSDs would catch up; that was only 30 years ago. Norman Wilson Toronto ON From dot at dotat.at Tue Mar 10 03:09:02 2020 From: dot at dotat.at (Tony Finch) Date: Mon, 9 Mar 2020 17:09:02 +0000 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: <1583708946.8301.for-standards-violators@oclsc.org> References: <1583708946.8301.for-standards-violators@oclsc.org> Message-ID: Norman Wilson wrote: > > I wish POSIX and Linux and the BSDs would catch up; that > was only 30 years ago. After sockets were added to BSD, pipe() was implemented using them. But it was a slightly different implementation than socketpair(). e.g. https://minnie.tuhs.org/cgi-bin/utree.pl?file=4.2BSD/usr/src/sys/sys/uipc_syscalls.c and the amusing comment at the top of https://minnie.tuhs.org/cgi-bin/utree.pl?file=4.2BSD/usr/src/sys/sys/uipc_pipe.c But the socket-backed pipes weren't bidirectional even though they could have been - see this fix in FreeBSD https://svnweb.freebsd.org/base/head/sys/kern/uipc_syscalls.c?r1=12843&r2=13146 Which did not last long because socket-backed pipe were soon replaced with a new (bidirectional) implementation https://svnweb.freebsd.org/base/head/sys/kern/sys_pipe.c?view=log&log_pagestart=200#rev13675 Tony. -- f.anthony.n.finch http://dotat.at/ Ardnamurchan Point to Cape Wrath: South or southwest 4 or 5, increasing 6 to gale 8, then veering west later. Rough or very rough, occasionally high in west. Rain or drizzle, then showers. Moderate or good, occasionally poor at first. From jpl.jpl at gmail.com Tue Mar 10 07:06:20 2020 From: jpl.jpl at gmail.com (John P. Linderman) Date: Mon, 9 Mar 2020 17:06:20 -0400 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200305020517.GA13872@wopr> <20200308052632.GD20478@eureka.lemis.com> <202003080532.0285WcWn1544496@darkstar.fourwinds.com> Message-ID: Nothing I'm aware of. I didn't mind throwing "tac" over the wall, because it was trivial, probably a couple hours work for me, under a minute for Ken. But the rsort source is not at all trivial, and still of potential value to AT&T. The source managed to get out as part of the "Hancock" project. I found a link in https://www.wired.com/2007/10/att-invents-pro/ but the page is gone. It probably didn't help that Wired titled the article *AT&T Invents Programming Language for Mass Surveillance* That's horse-pucky, akin to "Pitchfork makers invent device for spearing babies". I'm trying to track down a copy that was released publicly. I'm not hopeful. On Mon, Mar 9, 2020 at 11:28 AM Tyler Adams wrote: > Woah, this sounds really useful, is there anything like it today? > > On Sun, Mar 8, 2020, 16:32 John P. Linderman wrote: > >> In the "UNIX SYSTEM" issue of the BSTJ back in October of 1984, I >> suggested that it might be better, both for functionality *and* >> performance, to have a sort that only worked on records with a *single* >> key to be sorted *lexicographically*, and put all the complexity of >> dealing with native integers, dates, case-mapping, etc into a key-building >> front end. I wrote such a sort built around a radix sort. The sort >> itself sported very few options relating to record format (fixed-length, >> newline terminated, and header-based, where an ascii header identified >> record length, and, optionally, key position and key length), where to find >> the key in fixed-length and newline terminated records, merge-only, check >> sort order only, unique, strip off the sort key (to avoid the need for a >> post-process in many cases). Key-building was usually near-trivial using >> awk or perl or a few commands for tweaking native integer and floating >> point values so they would sort lexicographically. The sort was stable and >> blazingly fast. Some summer students once complained to me that I was >> messing up a paper they were writing because my external sort was faster >> than an internal qsort... the kind of complaint that warms one's heart. At >> the back of my mind was a generic key-building library that would >> accommodate (decimal) numbers of arbitrary length, with or without "E" >> exponents, dates in various formats, string collation for Unicode, etc. It >> remains at the back of my mind. >> >> On Sun, Mar 8, 2020 at 5:32 AM Tyler Adams wrote: >> >>> The idea of a simple rule is great, but the suggested rule fails on sort >>> -u which afaik came after sort | uniq for performance reasons. >>> >>> Another idea on the same vein is that a flag should be added only when >>> the job can be done inside the program and not with stdin/stdout (or no >>> flag can be added if one can reproduce the same behavior using pipelines). >>> >>> So, you need sort -u because only within sort can you get the >>> performance needed to get the job done. >>> >>> But you don't need -h in ls -lh. All the information to render a human >>> readable number is present on stdout of ls -l. You could easily have a >>> filter which renders numbers with options like adding commas, dots, >>> scientific notation, precision, money, units, etc. >>> >>> Tyler >>> >>> On Sun, Mar 8, 2020, 07:33 Jon Steinhart wrote: >>> >>>> After following this discussion, I guess that I have a simplistic way to >>>> determine whether something should be a dash option or a filter. In >>>> general, I'd make a filter if whatever it was doing was applicable to >>>> more than one command, a dash option otherwise. >>>> >>>> Jon >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From khm at sciops.net Tue Mar 10 07:22:57 2020 From: khm at sciops.net (Kurt H Maier) Date: Mon, 9 Mar 2020 14:22:57 -0700 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200305020517.GA13872@wopr> <20200308052632.GD20478@eureka.lemis.com> <202003080532.0285WcWn1544496@darkstar.fourwinds.com> Message-ID: <20200309212257.GB39634@wopr> On Mon, Mar 09, 2020 at 05:06:20PM -0400, John P. Linderman wrote: > but the page is gone. It probably didn't help that Wired titled the article > > *AT&T Invents Programming Language for Mass Surveillance* > > That's horse-pucky, akin to "Pitchfork makers invent device for spearing > babies". I'm trying to track down a copy that was released publicly. I'm > not hopeful. There is a copy here: https://github.com/mqudsi/hancock Not sure what other conclusion Wired was supposed to come to, given that the provided "Hello World" programs in the paper were all mass surveillance examples (tracking international calls to given numbers, tracking data streams to given IP addresses, and tracking specific connections to a given ISP). The license in the linked repository is different than the old password-gated NSL that was applied on the research.att.com pages. I wonder how many licenses this code was released with, over the years. khm From gtaylor at tnetconsulting.net Tue Mar 10 09:22:57 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Mon, 9 Mar 2020 17:22:57 -0600 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: <20200308151332.GA70333@clarinet.employees.org> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> Message-ID: <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> On 3/8/20 9:13 AM, Derek Fawcus wrote: > Now what would have been useful is a way to have distinct fd's for > the local read and write end of (e.g.) a TCP socket - such that > one direction could be closed w/o closing the other. I believe that this can be done, now. At least I've read that it's possible for one end to close (FIN) a TCP connection without the other end also closing. Thus you end up with the one-way data flow that is still ACKed the way that TCP does. -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From gtaylor at tnetconsulting.net Tue Mar 10 09:22:36 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Mon, 9 Mar 2020 17:22:36 -0600 Subject: [TUHS] First appearance of named pipes In-Reply-To: <20200306224431.D226C18C080@mercury.lcs.mit.edu> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> Message-ID: <1fab6a1a-ccfb-02cc-c23e-1a0d8c9e954a@spamtrap.tnetconsulting.net> On 3/6/20 3:44 PM, Noel Chiappa wrote: > a pipe is a FIFO. Hum.... Does that mean that I should be able to replace pipes in my shell commands with a collections of FIFOs? I'm not saying that I would want to. Well, at least not beyond an academic exercise. > RAND ports just allowed (effectively) a pipe to have a name in the > file system. I've got to admit that this statement, particularly the first two words, caused me a lot of trouble. The first few times I read it I kept thinking that RAND ported something from another platform / architecture. I now think it effectively means "(the) port (solution) from RAND". Suffice it to say that many of the emails in this thread didn't make sense with my first, seemingly incorrect, understanding, and made a LOT more sense with my second, seemingly correct, understanding. > The implementation of both is pretty straight-forward. A pipe is > just a file which has a maximum length, after which the writer is > blocked. A port is just a pipe (it uses the pipe code) whose inode > appears in the file system. Intriguing. It is interesting learning the history of things that I've taken for granted for 25 years. This thread aligns quite well with some external reading that I've done trying to learn more about file descriptors and redirection. TL;DR: The redirection symbols (syntax) weren't doing what I thought they were doing. I now believe that they change values of variables used for I/O. Much like DD statements in JCL. -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From lm at mcvoy.com Tue Mar 10 09:44:04 2020 From: lm at mcvoy.com (Larry McVoy) Date: Mon, 9 Mar 2020 16:44:04 -0700 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> Message-ID: <20200309234404.GV8730@mcvoy.com> On Mon, Mar 09, 2020 at 05:22:57PM -0600, Grant Taylor via TUHS wrote: > On 3/8/20 9:13 AM, Derek Fawcus wrote: > >Now what would have been useful is a way to have distinct fd's for the > >local read and write end of (e.g.) a TCP socket - such that one direction > >could be closed w/o closing the other. > > I believe that this can be done, now. At least I've read that it's possible > for one end to close (FIN) a TCP connection without the other end also > closing. Thus you end up with the one-way data flow that is still ACKed the > way that TCP does. Yes, it can be tricky to do it portably but I have. From arnold at skeeve.com Tue Mar 10 17:29:54 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Tue, 10 Mar 2020 01:29:54 -0600 Subject: [TUHS] First appearance of named pipes In-Reply-To: <1fab6a1a-ccfb-02cc-c23e-1a0d8c9e954a@spamtrap.tnetconsulting.net> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <1fab6a1a-ccfb-02cc-c23e-1a0d8c9e954a@spamtrap.tnetconsulting.net> Message-ID: <202003100729.02A7Tsg0002007@freefriends.org> Grant Taylor via TUHS wrote: > On 3/6/20 3:44 PM, Noel Chiappa wrote: > > a pipe is a FIFO. > > Hum.... > > Does that mean that I should be able to replace pipes in my shell > commands with a collections of FIFOs? Absolutely: $ mkfifo the_fifo $ ls -l the_fifo prw-rw-r-- 1 arnold arnold 0 Mar 10 09:28 the_fifo $ echo foo > the_fifo & sleep 1 ; cat the_fifo [1] 3721 foo [1]+ Done echo foo > the_fifo As you stated, not that you'd want to do that, but you can. Arnold From chet.ramey at case.edu Tue Mar 10 23:49:39 2020 From: chet.ramey at case.edu (Chet Ramey) Date: Tue, 10 Mar 2020 09:49:39 -0400 Subject: [TUHS] First appearance of named pipes In-Reply-To: <1fab6a1a-ccfb-02cc-c23e-1a0d8c9e954a@spamtrap.tnetconsulting.net> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <1fab6a1a-ccfb-02cc-c23e-1a0d8c9e954a@spamtrap.tnetconsulting.net> Message-ID: <178d5ef9-b188-ad63-2dc3-ff51541f6dc9@case.edu> On 3/9/20 7:22 PM, Grant Taylor via TUHS wrote: > On 3/6/20 3:44 PM, Noel Chiappa wrote: >> a pipe is a FIFO. > > Hum.... > > Does that mean that I should be able to replace pipes in my shell commands > with a collections of FIFOs?  Yes, absolutely. Just like a pipe is a FIFO, a FIFO is just a pipe with a name that appears in the file system. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ From doug at cs.dartmouth.edu Wed Mar 11 02:15:21 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Tue, 10 Mar 2020 12:15:21 -0400 Subject: [TUHS] Command line options and complexity Message-ID: <202003101615.02AGFLgS075920@coolidge.cs.dartmouth.edu> > The idea of a simple rule is great, but the suggested rule fails on sort -u > which afaik came after sort | uniq for performance reasons. As the guilty party for most of sort's comparison options, I can attest that efficiency was not an objective of -u. It was invented precisely because uniq had proved useful, but not when one was interested in uniqueness only of some key aspect of the data. -u differs from uniq in that -u selects samples based on equality of keys, not equality of lines. In the default case of whole-line keys, sort -u of course does exactly what sort|uniq does. For many applications of -u with keys, the non-key fields are not of interest. Then sed s/nonkeys//|sort|uniq may suffice. But sed did not exist when -u was invented. And not all sort key specs are easily imitated in sed. Doug From crossd at gmail.com Wed Mar 11 03:38:23 2020 From: crossd at gmail.com (Dan Cross) Date: Tue, 10 Mar 2020 13:38:23 -0400 Subject: [TUHS] Command line options and complexity In-Reply-To: <202003101615.02AGFLgS075920@coolidge.cs.dartmouth.edu> References: <202003101615.02AGFLgS075920@coolidge.cs.dartmouth.edu> Message-ID: On Tue, Mar 10, 2020 at 12:16 PM Doug McIlroy wrote: > > The idea of a simple rule is great, but the suggested rule fails on sort > -u > > which afaik came after sort | uniq for performance reasons. > > As the guilty party for most of sort's comparison options, I can > attest that efficiency was not an objective of -u. It was invented > precisely because uniq had proved useful, but not when one was > interested in uniqueness only of some key aspect of the data. > > -u differs from uniq in that -u selects samples based on > equality of keys, not equality of lines. In the default > case of whole-line keys, sort -u of course does exactly > what sort|uniq does. > > For many applications of -u with keys, the non-key fields > are not of interest. Then sed s/nonkeys//|sort|uniq may > suffice. But sed did not exist when -u was invented. > And not all sort key specs are easily imitated in sed. > This begs questions of stability: in the event of non-unique keys and non-key fields in the sortable data, which "records" (lines) are kept and which are discarded? Surely the "first" is kept and subsequent entries with the same key suppressed, but I confess I don't know enough about the internals of sed to know even what algorithm it uses (I assume a disk-based merge sort?), but I would imagine these details have changed over time. - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at bitblocks.com Wed Mar 11 03:44:32 2020 From: bakul at bitblocks.com (Bakul Shah) Date: Tue, 10 Mar 2020 10:44:32 -0700 Subject: [TUHS] Command line options and complexity In-Reply-To: Your message of "Tue, 10 Mar 2020 13:38:23 -0400." References: <202003101615.02AGFLgS075920@coolidge.cs.dartmouth.edu> Message-ID: <20200310174440.00B20156E45D@mail.bitblocks.com> On Tue, 10 Mar 2020 13:38:23 -0400 Dan Cross wrote: > > This begs questions of stability: in the event of non-unique keys and > non-key fields in the sortable data, which "records" (lines) are kept and > which are discarded? Surely the "first" is kept and subsequent entries with > the same key suppressed, but I confess I don't know enough about the > internals of sed to know even what algorithm it uses (I assume a disk-based > merge sort?), but I would imagine these details have changed over time. FreeBSD manpage for sort says that -u implies a stable sort, similar to -s. From crossd at gmail.com Wed Mar 11 04:09:39 2020 From: crossd at gmail.com (Dan Cross) Date: Tue, 10 Mar 2020 14:09:39 -0400 Subject: [TUHS] Command line options and complexity In-Reply-To: <20200310174440.00B20156E45D@mail.bitblocks.com> References: <202003101615.02AGFLgS075920@coolidge.cs.dartmouth.edu> <20200310174440.00B20156E45D@mail.bitblocks.com> Message-ID: On Tue, Mar 10, 2020 at 1:44 PM Bakul Shah wrote: > On Tue, 10 Mar 2020 13:38:23 -0400 Dan Cross wrote: > > > > This begs questions of stability: in the event of non-unique keys and > > non-key fields in the sortable data, which "records" (lines) are kept and > > which are discarded? Surely the "first" is kept and subsequent entries > with > > the same key suppressed, but I confess I don't know enough about the > > internals of sed to know even what algorithm it uses (I assume a > disk-based > > merge sort?), but I would imagine these details have changed over time. > > FreeBSD manpage for sort says that -u implies a stable sort, > similar to -s. > Thanks; that makes sense. I'm still interested in historical data, though. :-) - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug at cs.dartmouth.edu Wed Mar 11 04:42:53 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Tue, 10 Mar 2020 14:42:53 -0400 Subject: [TUHS] Command line options and complexity Message-ID: <202003101842.02AIgr2x082209@coolidge.cs.dartmouth.edu> > This begs questions of stability Astute question. I had that in my original draft, but eliminited it for what I thought was clarity. Anyway, depending on implementation of sort, you may need sort -s. Of course it doesn't matter which copy among several equal lines uniq produces, nor does it matter in sort when there are no comparison options--they're all the same. > I don't know enough about the > internals of sed to know even what algorithm it uses > (... a disk-based merge sort?) sed is not a sorting program--basically it copies input to output, making line-by-line editing changes. That's the way I meant to use it in sed s/nonkeys//|sort -keys|uniq. (I have added options to sort, hopefully for clarity). The argument to sed here means substitute the empty string for the nonkey fields (specified by a regular expression). If "sed" was a typo for "sort", all versions of sort that I know of use an internal sorting algorithm for big chunks of the file, then combines the chunks by merge. But internal sorting varies all over the map--variations on quicksort, radix sort, merge sort, ... Doug From crossd at gmail.com Wed Mar 11 05:38:50 2020 From: crossd at gmail.com (Dan Cross) Date: Tue, 10 Mar 2020 15:38:50 -0400 Subject: [TUHS] Command line options and complexity In-Reply-To: <202003101842.02AIgr2x082209@coolidge.cs.dartmouth.edu> References: <202003101842.02AIgr2x082209@coolidge.cs.dartmouth.edu> Message-ID: On Tue, Mar 10, 2020 at 2:43 PM Doug McIlroy wrote: > > This begs questions of stability > > Astute question. I had that in my original draft, but eliminited > it for what I thought was clarity. Anyway, depending on implementation > of sort, you may need sort -s. Of course it doesn't matter which copy > among several equal lines uniq produces, nor does it matter in sort > when there are no comparison options--they're all the same. > Thanks. That's interesting. Did `sort -s` come later? The idea that you preferred clarity over stability for `sort -u` would indicate so, otherwise one might imagine that `-u` would just imply `-s` and that would be that. > I don't know enough about the > > internals of sed to know even what algorithm it uses > > (... a disk-based merge sort?) > > sed is not a sorting program--basically it copies input to > output, making line-by-line editing changes. That's the > way I meant to use it in sed s/nonkeys//|sort -keys|uniq. > (I have added options to sort, hopefully for clarity). > The argument to sed here means substitute the empty > string for the nonkey fields (specified by a regular expression). > `sed` in my email was a typo, as you speculated below. Interestingly, this `sed` construction prior to `sort` loses information, which perhaps doesn't matter in any given specific case, but is insufficient in general, which I gathered to be the entire reason you implemented `sort -u`. If "sed" was a typo for "sort", It was. all versions of sort that > I know of use an internal sorting algorithm for big chunks > of the file, then combines the chunks by merge. But internal > sorting varies all over the map--variations on quicksort, > radix sort, merge sort, ... > It's the details of the internal sorts that are most interesting in some sense, as the merges are probably fairly straight forward but the internal sorts will affect stability and have other interesting characteristics. As an aside, one must imagine that, in this day and age, a "big chunk" is probably big enough to hold the vast majority of files entirely in RAM, and only exceptionally large files actually require merging multiple blocks. - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Wed Mar 11 06:26:10 2020 From: dave at horsfall.org (Dave Horsfall) Date: Wed, 11 Mar 2020 07:26:10 +1100 (EST) Subject: [TUHS] First appearance of named pipes In-Reply-To: <178d5ef9-b188-ad63-2dc3-ff51541f6dc9@case.edu> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <1fab6a1a-ccfb-02cc-c23e-1a0d8c9e954a@spamtrap.tnetconsulting.net> <178d5ef9-b188-ad63-2dc3-ff51541f6dc9@case.edu> Message-ID: On Tue, 10 Mar 2020, Chet Ramey wrote: > Yes, absolutely. Just like a pipe is a FIFO, a FIFO is just a pipe with > a name that appears in the file system. So they would still be subject to the same 8-block limit? I haven't delved into the finer points of named pipes as I rarely use them. Related question: are they the same as Unix-domain sockets? Oddly enough I haven't used those much either. -- Dave From chet.ramey at case.edu Wed Mar 11 06:37:14 2020 From: chet.ramey at case.edu (Chet Ramey) Date: Tue, 10 Mar 2020 16:37:14 -0400 Subject: [TUHS] First appearance of named pipes In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <1fab6a1a-ccfb-02cc-c23e-1a0d8c9e954a@spamtrap.tnetconsulting.net> <178d5ef9-b188-ad63-2dc3-ff51541f6dc9@case.edu> Message-ID: <82dba3bc-a514-c5e9-eefd-615b08f58cbf@case.edu> On 3/10/20 4:26 PM, Dave Horsfall wrote: > On Tue, 10 Mar 2020, Chet Ramey wrote: > >> Yes, absolutely. Just like a pipe is a FIFO, a FIFO is just a pipe with a >> name that appears in the file system. > > So they would still be subject to the same 8-block limit?  I haven't delved > into the finer points of named pipes as I rarely use them. I believe that on Linux, at least, they have the same capacity limits (64K), but you can modify that. I don't know about BSD. > Related question: are they the same as Unix-domain sockets?  Oddly enough I > haven't used those much either. They're similar, I guess ("I am but a shadowy reflection of you."). But I haven't used unix domain sockets, either. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ From drsalists at gmail.com Wed Mar 11 09:03:42 2020 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 10 Mar 2020 16:03:42 -0700 Subject: [TUHS] Command line options and complexity In-Reply-To: <202003031815.023IFSlD493028@darkstar.fourwinds.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> Message-ID: When I took a comparative languages class in school, the teacher said that the complexity of a programming language varies with the square of its number of features. I wonder if it's similar for command line options in shell-callables? On the other hand, adding command line options was (at least at one time) seen as a way of distinguishing GNU tools from Unix tools - that is, they were seen as a way of avoiding the copyright lawsuits that were snipping at BSD's heels. -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug at cs.dartmouth.edu Wed Mar 11 11:14:47 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Tue, 10 Mar 2020 21:14:47 -0400 Subject: [TUHS] eqn anomaly Message-ID: <202003110114.02B1ElO2029285@tahoe.cs.Dartmouth.EDU> I was surprised that eqn parses the macro call below as having two arguments, each with an unmatched ". .EQ define f % $1 % f("a,b") .EN Ralph Corderoy found that the comma can be hidden by replacing it with \N'44'. A somewhat cleaner way to hide it is .EQ define f % $1 % define comma % , % f(a comma b) .EN This works too. .EQ f(f(a comma b)) .EN [Note for cognoscenti. Eqn's practice in expanding macro arguments clashes with troff's. Eqn expands nested calls after substitution in the outer macro definition; troff expands while collecting arguments of the outer call. I've found no documentation of the eqn behavior.] The classical man page for eqn asserts categorically, `Strings enclosed in double quotes " " are passed through untouched.' Unfortunately the version of Kernighan/Cherry User's Guide that describes macros with arguments says little about how arguments are parsed except that they are separated by commas--nothing about whether commas are hidden by parentheses or quotes. Certainly splitting at a comma in a quoted string violates the plain meaning of the man-page assertion. If anyone has v10 (or perhaps something else after v7) running, I'd be grateful to learn what classic eqn actually did. I'm morally certain that if it did split and anyone had complained to Brian, he would have fixed it. These observations lead me to file a bug report. Doug From norman at oclsc.org Wed Mar 11 11:46:26 2020 From: norman at oclsc.org (Norman Wilson) Date: Tue, 10 Mar 2020 21:46:26 -0400 (EDT) Subject: [TUHS] eqn anomaly Message-ID: <20200311014626.2FC814422F@lignose.oclsc.org> On (post-)V10: echo '.EQ define f % $1 % f("a,b") .EN' | eqn emits .lf 1 - .EQ .ds 11 "\f2a,b\fP .if 1m>\n(.v .ne 1m .rn 11 10 \&\*(10 .EN .lf 5 On a Linux system with GNU eqn (groff) version 1.22.3, the output is rather more verbose (48 lines!), but the troff result is just an a (rather than the proper a,b) and eqn complains eqn::3: newline before end of quoted text I assume this Linux result is more or less what Doug expects. Norman Wilson Toronto ON (still heating my basement with a MicroVAX) From gtaylor at tnetconsulting.net Wed Mar 11 12:47:39 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 10 Mar 2020 20:47:39 -0600 Subject: [TUHS] First appearance of named pipes In-Reply-To: <202003100729.02A7Tsg0002007@freefriends.org> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <1fab6a1a-ccfb-02cc-c23e-1a0d8c9e954a@spamtrap.tnetconsulting.net> <202003100729.02A7Tsg0002007@freefriends.org> Message-ID: <4fb5b7ab-03a4-df64-d25b-8be587b6a17b@spamtrap.tnetconsulting.net> On 3/10/20 1:29 AM, arnold at skeeve.com wrote: > Absolutely: > > $ mkfifo the_fifo > $ ls -l the_fifo > prw-rw-r-- 1 arnold arnold 0 Mar 10 09:28 the_fifo > $ echo foo > the_fifo & sleep 1 ; cat the_fifo > [1] 3721 > foo > [1]+ Done echo foo > the_fifo > > As you stated, not that you'd want to do that, but you can. Thank you for your reply Arnold. As I was reading your reply, I realized that I did not fully convey the question that I was still mulling over in my head. (More in a moment.) This thread is one of about three things happening in my life that have to do with pipes, FIFOs, and file descriptors. I managed to articulate the simpler of the questions while reading Noel's email. The larger more onerous question is could I leverage exec to alter where file descriptors 0 (STDIN), 1 (STDOUT), and 2 (STDERR) are set to, including changing 1 to the value of a FIFO, and 0 of a subsequent command to also be the value of the FIFO, thus have pipe like behavior between two commands without using a pipe or redirection as in ">". This has also gotten me to wonder about the possibility of having multiple commands output to a file descriptor; 1 / 2 / other, that is input to a separate command. Sort of the opposite of tee, in a manner of speaking. I'll try to articulate: $ mkfifo test.fifo $ exec 3>&1 $ exec 1> test.fifo $ for l in {a..z}; do echo $l; sleep 1; done & $ for L in {A..Z}; do echo $L; sleep 1; done & $ for n in {1..100}; do echo $n; sleep 1; done & $ exec 1>&3 $ cat test.fifo This seems special to me in that I have three processes (for loops) writing into what is effectively the same pipe. After having mulled this over for a few days and typing this out, I realize that the "pipe" is really just a fifo and that in this case the fifo is a named pipe on the file system. I could do the same thing with a file. Historically I would have done the same thing with a file. But now I realize that the file is not required and that I can use a fifo which is in memory and never hits the disk. (Save for creating the name interface to the pipe / fifo.) At least, I think that's all accurate. I would be very eager to learn from anyone who is willing to teach me pointers. :-) -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From gtaylor at tnetconsulting.net Wed Mar 11 12:51:08 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Tue, 10 Mar 2020 20:51:08 -0600 Subject: [TUHS] First appearance of named pipes In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <1fab6a1a-ccfb-02cc-c23e-1a0d8c9e954a@spamtrap.tnetconsulting.net> <178d5ef9-b188-ad63-2dc3-ff51541f6dc9@case.edu> Message-ID: <74383b50-8f50-2199-e12d-d8e968fb3fd5@spamtrap.tnetconsulting.net> On 3/10/20 2:26 PM, Dave Horsfall wrote: > Related question: are they the same as Unix-domain sockets? Prior to this thread, I would have said that named pipes / FIFOs are unidirectional and that sockets are bi-directional. But there have been a number of things in this thread that call that understanding into question. Now I need to mull things over and get a new working understanding. -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From dave at horsfall.org Wed Mar 11 13:18:08 2020 From: dave at horsfall.org (Dave Horsfall) Date: Wed, 11 Mar 2020 14:18:08 +1100 (EST) Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> Message-ID: On Tue, 10 Mar 2020, Dan Stromberg wrote: > When I took a comparative languages class in school, the teacher said > that the complexity of a programming language varies with the square of > its number of features. That sort of makes sense from a mathematical point of view, if you regard it as a matrix of side effects. I hate to think about how it affects Perl (my favourite language) though :-) > I wonder if it's similar for command line options in shell-callables? I'm starting to think that if a utility requires many options then perhaps they ought to be split into filters (or at least environment variables); I despair at how *ix is drifting from "one tool, one job" to "one size fits all"... The "ls" command for example really needs an option-ectomy; I find that I don't really care about the exact number of bytes there are in a file as the nearest KiB or MiB (or even GiB) is usually good enough, so I'd be happy if "-h" was the default with some way to turn it off (yes, I know that it's occasionally useful to add them all up in a column, but that won't tell you how many media blocks are required). Quickly now, without looking: which option shows unprintable characters in a filename? Unless you use it regularly (in which case you have real problems) you would have to look it up; I find that "ls ... | od -bc" to be quicker, especially on filenames with trailing blanks etc (which "-B" won't show). > On the other hand, adding command line options was (at least at one > time) seen seen as a way of distinguishing GNU tools from Unix tools - > that is, they were seen as a way of avoiding the copyright lawsuits that > were snipping at BSD's heels. I've never liked GNU's "--bloody-long-option" convention as you still have to look up which one does what, but I've never thought about that view; a lot of long options still accept a single character (subject to feeping creaturism, of course). -- Dave From usotsuki at buric.co Wed Mar 11 14:02:29 2020 From: usotsuki at buric.co (Steve Nickolas) Date: Wed, 11 Mar 2020 00:02:29 -0400 (EDT) Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> Message-ID: On Wed, 11 Mar 2020, Dave Horsfall wrote: > I'm starting to think that if a utility requires many options then perhaps > they ought to be split into filters (or at least environment variables); I > despair at how *ix is drifting from "one tool, one job" to "one size fits > all"... > > The "ls" command for example really needs an option-ectomy; I find that I > don't really care about the exact number of bytes there are in a file as the > nearest KiB or MiB (or even GiB) is usually good enough, so I'd be happy if > "-h" was the default with some way to turn it off (yes, I know that it's > occasionally useful to add them all up in a column, but that won't tell you > how many media blocks are required). > > Quickly now, without looking: which option shows unprintable characters in a > filename? Unless you use it regularly (in which case you have real problems) > you would have to look it up; I find that "ls ... | od -bc" to be quicker, > especially on filenames with trailing blanks etc (which "-B" won't show). It would probably be interesting to define a simplified standard, because yeesh, trying to implement even a command as basic as ls is just torture (mainly because it basically requires putting all of "column" and most of "sort" into it)! > I've never liked GNU's "--bloody-long-option" convention as you still have to > look up which one does what, but I've never thought about that view; a lot of > long options still accept a single character (subject to feeping creaturism, > of course). I'm still into the one-character switch thing, personally. -uso. From arnold at skeeve.com Wed Mar 11 17:02:59 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Wed, 11 Mar 2020 01:02:59 -0600 Subject: [TUHS] First appearance of named pipes In-Reply-To: <4fb5b7ab-03a4-df64-d25b-8be587b6a17b@spamtrap.tnetconsulting.net> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <1fab6a1a-ccfb-02cc-c23e-1a0d8c9e954a@spamtrap.tnetconsulting.net> <202003100729.02A7Tsg0002007@freefriends.org> <4fb5b7ab-03a4-df64-d25b-8be587b6a17b@spamtrap.tnetconsulting.net> Message-ID: <202003110702.02B72xqj023433@freefriends.org> Hi Grant. Grant Taylor via TUHS wrote: > The larger more onerous question is could I leverage exec to alter where > file descriptors 0 (STDIN), 1 (STDOUT), and 2 (STDERR) are set to, > including changing 1 to the value of a FIFO, and 0 of a subsequent > command to also be the value of the FIFO, thus have pipe like behavior > between two commands without using a pipe or redirection as in ">". There's nothing preventing you from doing that. After the fork() and before the exec(), just close() and dup() the relevant fds in the right order and you're set. > This has also gotten me to wonder about the possibility of having > multiple commands output to a file descriptor; 1 / 2 / other, that is > input to a separate command. Sort of the opposite of tee, in a manner > of speaking. I'll try to articulate: > > $ mkfifo test.fifo > $ exec 3>&1 > $ exec 1> test.fifo > $ for l in {a..z}; do echo $l; sleep 1; done & > $ for L in {A..Z}; do echo $L; sleep 1; done & > $ for n in {1..100}; do echo $n; sleep 1; done & > $ exec 1>&3 > $ cat test.fifo I don't think that this is any different from: (for l in {a..z}; do echo $l; sleep 1; done & for L in {A..Z}; do echo $L; sleep 1; done & for n in {1..100}; do echo $n; sleep 1; done &) | cat which reduces to: (for l in {a..z}; do echo $l; sleep 1; done & for L in {A..Z}; do echo $L; sleep 1; done & for n in {1..100}; do echo $n; sleep 1; done &) > /some/file (You might want to background that whole mess given that the final pipeline will sleep for 100 seconds.) > This seems special to me in that I have three processes (for loops) > writing into what is effectively the same pipe. It's not any different than calling `stty -tostop' and then simply backgrounding the three loops at the terminal. Try it! (This is the beauty of the Unix model, an fd is just a data sink, we don't care where it goes.) > I would be very eager to learn from anyone who is willing to teach me > pointers. :-) At the cost of tooting my own horn, I recommend my book "Linux Programming by Example: The Fundamentals", which, despite the "Linux" in the title, covers basic Unix programming, including file descriptor manipulation of the sort under discussion here. HTH, Arnold From jpl.jpl at gmail.com Thu Mar 12 03:41:48 2020 From: jpl.jpl at gmail.com (John P. Linderman) Date: Wed, 11 Mar 2020 13:41:48 -0400 Subject: [TUHS] Command line options and complexity In-Reply-To: <20200309212257.GB39634@wopr> References: <20200305020517.GA13872@wopr> <20200308052632.GD20478@eureka.lemis.com> <202003080532.0285WcWn1544496@darkstar.fourwinds.com> <20200309212257.GB39634@wopr> Message-ID: This is *great*, Kurt. The source in src/runtime/hrs/src for rsort.c is their version of my external sort, modified to be a subroutine. There's some lessons to be learned about "software hygiene". I was cavalier about freeing what I allocated dynamically. As a result, their version leaks like a sieve if the subroutine is called repeatedly. Apropos of which, they came to me having noted that only the first call was acting as expected. There's a wonderful irony (I'm big on irony). I had replaced my do-it-yourself argument processing with getopt. The code has the following comment ** Use getopt() for portability. A few lines later, you see optind = 1; /* reset after use in Hancock program * while ((c = getopt(argc, argv, "cCiIjmrsSuvb:f:D:o:p:T:x:y:z:")) != EOF) { optind??? Seems getopt has an undocumented global flag to prevent reprocessing the arguments. How portable:-) Anyway, it should be possible to turn rsort.c back into standalone code. I'd be the obvious person to do it, but that would probably be a violation of some agreement with AT&T. However, if somebody else wants to take on the task (it would make a great summer intern project), I'd be happy to share ideas I have had since retiring that would improve the code. fc.c in the same directory is a library-ized version of a fixcut command I wrote as a fixed-length counterpart to the cut command, for fixed-length inputs (like native floats and integers, which can be tweaked to sort lexicographically). Unlike rsort, I practiced good hygiene and kept track of all allocated space so it could be freed. Too bad they didn't include the man pages for rsort and fixcut. They'd make it easier to understand them. Jon Bentley observed that "comments are love letters to your future self", and I feel a lot of love from the heavily commented rsort code. This probably should move to coff, it's not really about UNIX history (although rsort has vestigial traces of ancient days, like the code to write checkpoint files after each output temp is closed... sorting a million bytes once took hours, with slow processors and disks. It was painful to have to start from scratch if an overnight sort got interrupted. Now sorting a billion bytes is pretty quick, and the checkpoint stuff never gets used. It's one of the things that could profitably disappear.) On Mon, Mar 9, 2020 at 5:22 PM Kurt H Maier wrote: > On Mon, Mar 09, 2020 at 05:06:20PM -0400, John P. Linderman wrote: > > but the page is gone. It probably didn't help that Wired titled the > article > > > > *AT&T Invents Programming Language for Mass Surveillance* > > > > That's horse-pucky, akin to "Pitchfork makers invent device for spearing > > babies". I'm trying to track down a copy that was released publicly. I'm > > not hopeful. > > There is a copy here: https://github.com/mqudsi/hancock > > Not sure what other conclusion Wired was supposed to come to, given that > the provided "Hello World" programs in the paper were all mass > surveillance examples (tracking international calls to given numbers, > tracking data streams to given IP addresses, and tracking specific > connections to a given ISP). > > The license in the linked repository is different than the old > password-gated NSL that was applied on the research.att.com pages. I > wonder how many licenses this code was released with, over the years. > > > khm > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Thu Mar 12 07:29:05 2020 From: imp at bsdimp.com (Warner Losh) Date: Wed, 11 Mar 2020 15:29:05 -0600 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200305020517.GA13872@wopr> <20200308052632.GD20478@eureka.lemis.com> <202003080532.0285WcWn1544496@darkstar.fourwinds.com> <20200309212257.GB39634@wopr> Message-ID: On Wed, Mar 11, 2020 at 11:43 AM John P. Linderman wrote: > This is *great*, Kurt. The source in src/runtime/hrs/src for rsort.c is > their version of my external sort, modified to be a subroutine. There's > some lessons to be learned about "software hygiene". I was cavalier about > freeing what I allocated dynamically. As a result, their version leaks like > a sieve if the subroutine is called repeatedly. Apropos of which, they came > to me having noted that only the first call was acting as expected. There's > a wonderful irony (I'm big on irony). I had replaced my do-it-yourself > argument processing with getopt. The code has the following comment > > ** Use getopt() for portability. > > A few lines later, you see > > optind = 1; /* reset after use in Hancock program * > while ((c = getopt(argc, argv, "cCiIjmrsSuvb:f:D:o:p:T:x:y:z:")) != > EOF) { > > optind??? Seems getopt has an undocumented global flag to prevent > reprocessing the arguments. How portable:-) > It's documented: The variables opterr and optind are both initialized to 1. The optind variable may be set to another value before a set of calls to getopt() in order to skip over more or less argv entries. is what the FreeBSD man page has to say about it. So this just resets any scanning that had happened before this... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From grog at lemis.com Thu Mar 12 08:56:38 2020 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Thu, 12 Mar 2020 09:56:38 +1100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> Message-ID: <20200311225638.GG89512@eureka.lemis.com> On Wednesday, 11 March 2020 at 14:18:08 +1100, Dave Horsfall wrote: > > The "ls" command for example really needs an option-ectomy; I find that I > don't really care about the exact number of bytes there are in a file as > the nearest KiB or MiB (or even GiB) is usually good enough, so I'd be > happy if "-h" was the default with some way to turn it off (yes, I know > that it's occasionally useful to add them all up in a column, but that > won't tell you how many media blocks are required). A good example. But you're not removing options, you're just redefining them. In fact I find the -h option particularly emetic, so a better choice in removing options would be to remove -h and use a filter to mutilate the sizes: $ ls -l | humanize But that's a pain, isn't it? That's why there's a -h option for people who like it. Note that you can't do it the other way round: you can't get the exact size from -h output. And then there's the question why you don't like the standard output. Because the number strings are too long and difficult to read, maybe? That's the rationale for the -, option. > Quickly now, without looking: which option shows unprintable > characters in a filename? Unless you use it regularly (in which > case you have real problems) you would have to look it up; I find > that "ls ... | od -bc" to be quicker, especially on filenames with > trailing blanks etc (which "-B" won't show). This is arguably a bug in the -B option. I certainly don't think the pipe notation is quicker. But it's nice to have both alternatives. Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From crossd at gmail.com Thu Mar 12 09:14:32 2020 From: crossd at gmail.com (Dan Cross) Date: Wed, 11 Mar 2020 19:14:32 -0400 Subject: [TUHS] Command line options and complexity In-Reply-To: <20200311225638.GG89512@eureka.lemis.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> Message-ID: On Wed, Mar 11, 2020 at 6:57 PM Greg 'groggy' Lehey wrote: > On Wednesday, 11 March 2020 at 14:18:08 +1100, Dave Horsfall wrote: > > > > The "ls" command for example really needs an option-ectomy; I find that I > > don't really care about the exact number of bytes there are in a file as > > the nearest KiB or MiB (or even GiB) is usually good enough, so I'd be > > happy if "-h" was the default with some way to turn it off (yes, I know > > that it's occasionally useful to add them all up in a column, but that > > won't tell you how many media blocks are required). > > A good example. But you're not removing options, you're just > redefining them. In fact I find the -h option particularly emetic, so > a better choice in removing options would be to remove -h and use a > filter to mutilate the sizes: > > $ ls -l | humanize > > But that's a pain, isn't it? I don't know; that's subjective. > That's why there's a -h option for > people who like it. That's incomplete, in that it implies that an option is the only way to achieve the goal of reducing the perceived pain, but that's not the case. (Note I'm not saying you intended that as an interpretation, but it's a reasonable intuition for an intention.) An interesting counterpoint to this argument is how columnized "ls" is handled under Plan 9: there is no `-C` option to `ls` there; instead, there's a general-purpose `mc` filter that figures out the size of the window it's running in, reads its input, decides how many columns the input will fit into, and emits it columnized. But yes, it would be a pain to type `ls | mc` every time one wanted columnized `ls` output, so this is wrapped up into a shell script called `lc`. Note that this lets you do stuff like, `lc -l` and see multi-column long listings if the window is wide enough. I got so used to this from plan9 that I keep an approximation in $HOME/bin/lc: `exec ls -ACF "$@"`. For the `humanize` thing, I don't see why one couldn't have an `lh` command that generated "human-friendly long output from ls." > Note that you can't do it the other way round: > you can't get the exact size from -h output. > That's true, but now the logic is specialized to ls, and not applicable to anything else (e.g., du? df? wc, perhaps?). Similarly with `-,`. It is not general purpose, which is unfortunate. Granted, combining these things would be a little challenging, but is it likely that one would want `ls -l,h`? Optimize for the common case, etc.... And then there's the question why you don't like the standard output. > Because the number strings are too long and difficult to read, maybe? > That's the rationale for the -, option. > > > Quickly now, without looking: which option shows unprintable > > characters in a filename? Unless you use it regularly (in which > > case you have real problems) you would have to look it up; I find > > that "ls ... | od -bc" to be quicker, especially on filenames with > > trailing blanks etc (which "-B" won't show). > > This is arguably a bug in the -B option. I certainly don't think the > pipe notation is quicker. But it's nice to have both alternatives. By default, plan9 would quote filenames that had characters that were special to the shell (there wasn't really the concept of "non-printable characters in the Unix/TTY sense); this could be disabled by specifying the `-Q` option. - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jpl.jpl at gmail.com Thu Mar 12 10:13:11 2020 From: jpl.jpl at gmail.com (John P. Linderman) Date: Wed, 11 Mar 2020 20:13:11 -0400 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200305020517.GA13872@wopr> <20200308052632.GD20478@eureka.lemis.com> <202003080532.0285WcWn1544496@darkstar.fourwinds.com> <20200309212257.GB39634@wopr> Message-ID: I wasn't running FreeBSD. Linux has nothing to say about it. The wonderful thing about standards is that there are so many to choose from. On Wed, Mar 11, 2020 at 5:29 PM Warner Losh wrote: > > > On Wed, Mar 11, 2020 at 11:43 AM John P. Linderman > wrote: > >> This is *great*, Kurt. The source in src/runtime/hrs/src for rsort.c is >> their version of my external sort, modified to be a subroutine. There's >> some lessons to be learned about "software hygiene". I was cavalier about >> freeing what I allocated dynamically. As a result, their version leaks like >> a sieve if the subroutine is called repeatedly. Apropos of which, they came >> to me having noted that only the first call was acting as expected. There's >> a wonderful irony (I'm big on irony). I had replaced my do-it-yourself >> argument processing with getopt. The code has the following comment >> >> ** Use getopt() for portability. >> >> A few lines later, you see >> >> optind = 1; /* reset after use in Hancock program * >> while ((c = getopt(argc, argv, "cCiIjmrsSuvb:f:D:o:p:T:x:y:z:")) != >> EOF) { >> >> optind??? Seems getopt has an undocumented global flag to prevent >> reprocessing the arguments. How portable:-) >> > > It's documented: > > The variables opterr and optind are both initialized to 1. The optind > variable may be set to another value before a set of calls to > getopt() in > order to skip over more or less argv entries. > > is what the FreeBSD man page has to say about it. So this just resets any > scanning that had happened before this... > > Warner > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chet.ramey at case.edu Thu Mar 12 10:34:24 2020 From: chet.ramey at case.edu (Chet Ramey) Date: Wed, 11 Mar 2020 20:34:24 -0400 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200305020517.GA13872@wopr> <20200308052632.GD20478@eureka.lemis.com> <202003080532.0285WcWn1544496@darkstar.fourwinds.com> <20200309212257.GB39634@wopr> Message-ID: <1b7198a7-3b41-6514-cdaa-b6eb152a61b8@case.edu> On 3/11/20 8:13 PM, John P. Linderman wrote: > I wasn't running FreeBSD. Linux has nothing to say about it. The wonderful > thing about standards is that there are so many to choose from. Did somebody mention ... standards? https://pubs.opengroup.org/onlinepubs/9699919799/functions/getopt.html#tag_16_206 -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ From grog at lemis.com Thu Mar 12 10:42:57 2020 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Thu, 12 Mar 2020 11:42:57 +1100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> Message-ID: <20200312004257.GH89512@eureka.lemis.com> On Wednesday, 11 March 2020 at 19:14:32 -0400, Dan Cross wrote: > On Wed, Mar 11, 2020 at 6:57 PM Greg 'groggy' Lehey wrote: > >> On Wednesday, 11 March 2020 at 14:18:08 +1100, Dave Horsfall wrote: >>> >>> The "ls" command for example really needs an option-ectomy; I find that I >>> don't really care about the exact number of bytes there are in a file as >>> the nearest KiB or MiB (or even GiB) is usually good enough, so I'd be >>> happy if "-h" was the default with some way to turn it off (yes, I know >>> that it's occasionally useful to add them all up in a column, but that >>> won't tell you how many media blocks are required). >> >> A good example. But you're not removing options, you're just >> redefining them. In fact I find the -h option particularly emetic, so >> a better choice in removing options would be to remove -h and use a >> filter to mutilate the sizes: >> >> $ ls -l | humanize >> >> But that's a pain, isn't it? > > I don't know; that's subjective. It's certainly more work than -h. >> That's why there's a -h option for people who like it. > > That's incomplete, in that it implies that an option is the only way > to achieve the goal of reducing the perceived pain, but that's not > the case. (Note I'm not saying you intended that as an > interpretation, but it's a reasonable intuition for an intention.) What I meant (and this is certainly my interpretation) was that somebody added the -h option because of perceived pain with piping output through another program. I didn't intend to imply that it was the only alternative. > An interesting counterpoint to this argument is how columnized "ls" > is handled under Plan 9: there is no `-C` option to `ls` there; > instead, there's a general-purpose `mc` filter that figures out the > size of the window it's running in, reads its input, decides how > many columns the input will fit into, and emits it columnized. But > yes, it would be a pain to type `ls | mc` every time one wanted > columnized `ls` output, so this is wrapped up into a shell script > called `lc`. Note that this lets you do stuff like, `lc -l` and see > multi-column long listings if the window is wide enough. Yes, that sounds like an excellent method. > For the `humanize` thing, I don't see why one couldn't have an `lh` > command that generated "human-friendly long output from ls." And yes, I deliberately didn't mention this option, though it occurred to me. I have a couple of scripts like this, like: alias l="ls -lbL," >> Note that you can't do it the other way round: you can't get the >> exact size from -h output. > > That's true, but now the logic is specialized to ls, and not > applicable to anything else (e.g., du? df? wc, perhaps?). Similarly > with `-,`. It is not general purpose, which is unfortunate. Yes, this is an issue that I mentioned in an earlier message (I added a positional parameter to work around it). But this is in the nature of the output. mc doesn't have this issue. > Granted, combining these things would be a little challenging, but is it > likely that one would want `ls -l,h`? Optimize for the common case, > etc.... Heh. Never thought of that. But since -h (apparently) never produces output with 4 digits, the -, doesn't ever come into effect. I've just tried it on some big files, and the -, is effectively ignored. > And then there's the question why you don't like the standard > output. I don't like the standard output because things like this are hard to read: -rw-r--r-- 1 grog lemis 8234010624 22 Mar 2012 Casanova-TV-1-5 -rw-r--r-- 1 grog home 13225168900 31 Aug 2019 Movie:_Sahara_2005-2016-04-11-2028 I find this easier to read: -rw-r--r-- 1 grog lemis 8,234,010,624 22 Mar 2012 Casanova-TV-1-5 -rw-r--r-- 1 grog home 13,225,168,900 31 Aug 2019 Movie:_Sahara_2005-2016-04-11-2028 I can't speak for Dave, but this is also less painful: -rw-r--r-- 1 grog lemis 7.7G 22 Mar 2012 Casanova-TV-1-5 -rw-r--r-- 1 grog home 12G 31 Aug 2019 Movie:_Sahara_2005-2016-04-11-2028 The problem for me there is the difficulty comparing lengths, and the implicit inaccuracy. >> Because the number strings are too long and difficult to read, maybe? >> That's the rationale for the -, option. >> >>> Quickly now, without looking: which option shows unprintable >>> characters in a filename? Unless you use it regularly (in which >>> case you have real problems) you would have to look it up; I find >>> that "ls ... | od -bc" to be quicker, especially on filenames with >>> trailing blanks etc (which "-B" won't show). >> >> This is arguably a bug in the -B option. I certainly don't think the >> pipe notation is quicker. But it's nice to have both alternatives. > > By default, plan9 would quote filenames that had characters that > were special to the shell (there wasn't really the concept of > "non-printable characters in the Unix/TTY sense); this could be > disabled by specifying the `-Q` option. Hmm. In this particular case, so does Linux: === grog at bilbo (/dev/pts/11) ~ 2 -> touch "foo " === grog at bilbo (/dev/pts/11) ~ 4 -> l foo* -rw-r--r-- 1 grog grog 1499570 Jun 30 2012 foo -rw-r--r-- 1 grog grog 0 Mar 12 10:40 'foo ' I wonder if that's something we should emulate in FreeBSD. At the very least we should consider whether the lack of identification of trailing blanks is a bug in the FreeBSD implementation of -B. This option isn't in POSIX, and in Linux it means -B, --ignore-backups do not list implied entries ending with ~ So maybe it's a candidate for fixing. Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From usotsuki at buric.co Thu Mar 12 10:53:12 2020 From: usotsuki at buric.co (Steve Nickolas) Date: Wed, 11 Mar 2020 20:53:12 -0400 (EDT) Subject: [TUHS] Command line options and complexity In-Reply-To: <20200311225638.GG89512@eureka.lemis.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> Message-ID: On Thu, 12 Mar 2020, Greg 'groggy' Lehey wrote: > On Wednesday, 11 March 2020 at 14:18:08 +1100, Dave Horsfall wrote: >> >> The "ls" command for example really needs an option-ectomy; I find that I >> don't really care about the exact number of bytes there are in a file as >> the nearest KiB or MiB (or even GiB) is usually good enough, so I'd be >> happy if "-h" was the default with some way to turn it off (yes, I know >> that it's occasionally useful to add them all up in a column, but that >> won't tell you how many media blocks are required). > > A good example. But you're not removing options, you're just > redefining them. In fact I find the -h option particularly emetic, so > a better choice in removing options would be to remove -h and use a > filter to mutilate the sizes: > > $ ls -l | humanize > > But that's a pain, isn't it? That's why there's a -h option for > people who like it. Note that you can't do it the other way round: > you can't get the exact size from -h output. > > And then there's the question why you don't like the standard output. > Because the number strings are too long and difficult to read, maybe? > That's the rationale for the -, option. > >> Quickly now, without looking: which option shows unprintable >> characters in a filename? Unless you use it regularly (in which >> case you have real problems) you would have to look it up; I find >> that "ls ... | od -bc" to be quicker, especially on filenames with >> trailing blanks etc (which "-B" won't show). > > This is arguably a bug in the -B option. I certainly don't think the > pipe notation is quicker. But it's nice to have both alternatives. > > Greg > -- > Sent from my desktop computer. > Finger grog at lemis.com for PGP public key. > See complete headers for address and phone numbers. > This message is digitally signed. If your Microsoft mail program > reports problems, please read http://lemis.com/broken-MUA > I went through all the switches defined by POSIX, and figured that those 26 could be cut down. My concept reduced the number of switches from 26 to 9 (FLRadfiln). Of course, the idea is to be more minimalist than POSIX, so some people's opinions on what is or isn't necessary may differ from mine. Of course, this changes the default behavior of ls because it no longer would be able to do columnar listings (|column for that). I felt -A was a redundant "almost -a". I felt -C and -x were redundant because a tool like column(1) could be used to do the same job (even though column(1) isn't POSIX). I felt -H was a redundant "almost -L". I felt -S, -r and -t could be implemented in other ways using sort(1). I felt -c and -u were meaningless, but that's because of the filesystems I usually work with that do not have functional equivalents. -u for one is completely useless on VFAT even though it has such timestamps! YMMV. I felt -g and -o could be replaced by cut(1). I felt -k wasn't really all that important. Just halve the numbers. I felt -m wasn't really all that important. There's other ways to convert to that format, no doubt, through filters. I felt -p was a redundant "almost -F". I felt -q could be done just fine with something like tr(1). I felt -s was a redundant "kindasorta -l". And -1 becomes the new default, so it's redundant. ;) Again, YMMV. ;) -uso. From grog at lemis.com Thu Mar 12 13:09:17 2020 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Thu, 12 Mar 2020 14:09:17 +1100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> Message-ID: <20200312030917.GJ89512@eureka.lemis.com> On Wednesday, 11 March 2020 at 20:53:12 -0400, Steve Nickolas wrote: > I went through all the switches defined by POSIX, and figured that > those 26 could be cut down. A brave man to defy POSIX! I wasn't so brave, which is why we have the -y option. > My concept reduced the number of switches from 26 to 9 (FLRadfiln). > Of course, the idea is to be more minimalist than POSIX, so some > people's opinions on what is or isn't necessary may differ from > mine. OK, let's compare notes: > I felt -A was a redundant "almost -a". Arguably -a could go too. The distinction seems arbitrary. > I felt -C and -x were redundant because a tool like column(1) could be > used to do the same job (even though column(1) isn't POSIX). Neither would this ls(1) be. > I felt -H was a redundant "almost -L". No arguments, but I suspect that somebody had a good reason for this distinction, and removing it could cause problems. > I felt -S, -r and -t could be implemented in other ways using sort(1). -S isn't POSIX. And to implement it without an option would mean removing -h. As I mentioned earlier, -t can't be done by a filter without significantly modifying the timestamp output. That was my rationale for the -D option, which allows sorting by an external filter. -r could work. > I felt -c and -u were meaningless, but that's because of the filesystems I > usually work with that do not have functional equivalents. -u for one is > completely useless on VFAT even though it has such timestamps! YMMV. I think this says more about your file systems than about the options. I find both incredibly useful, and there's no easy way to get the information elsewhere. stat(1) would be an option, but then that could replace ls(1) completely. > I felt -g and -o could be replaced by cut(1). -g is already obsolete in FreeBSD (accepted and ignored). -o has already been repurposed (show file flags). > I felt -k wasn't really all that important. Just halve the numbers. Agreed. > I felt -m wasn't really all that important. There's other ways to convert > to that format, no doubt, through filters. Possibly. Certainly I wouldn't miss it. > I felt -p was a redundant "almost -F". OK. > I felt -q could be done just fine with something like tr(1). I think that it could be replaced by -b. "?" isn't really very helpful. > I felt -s was a redundant "kindasorta -l". I can't agree with that, but I've never used it. The only sensible use would appear to be talking about disk blocks, but on FreeBSD at any rate it looks at the BLOCKSIZE environment variable, which I have set to 1048576 (so that utilities will display in MB where appropriate), and that's what -s does too: 2079 -rw-r--r-- 1 grog wheel 2,178,735,915 4 Oct 11:15 Willkommen-bei-den-Honeckers---Spielfilm,-Deutschland-2016-20191003-125200.mp4 That makes it pretty useless. So, any others? -G: Colorized output. I'd be *really* happy to get rid of this, but it's not easy to instate with a filter, so I suppose there are enough people who like it that it will have to stay. -P: Seems only to be there to cancel a -H or -L. -W: "Display whiteouts when scanning directories". I don't even understand what that is. -a: See discussion of -A. --color: Again, no thanks. -f: We haven't really discussed this one. If you want to remove -S, -r and -t, then arguably -f should become the default and be -removed. -n: Make it the default and require a filter to convert group and user numbers to IDs. -y: If we get rid of all sorting, it will no longer be needed. -,: Make the option standard: output numbers with commas every 3 digits. Then this option specification wouldn't be needed. Of course, none of this will happen. But it is interesting to think about it. In particular, options like -g and -o, which are no longer modern. Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From usotsuki at buric.co Thu Mar 12 13:34:46 2020 From: usotsuki at buric.co (Steve Nickolas) Date: Wed, 11 Mar 2020 23:34:46 -0400 (EDT) Subject: [TUHS] Command line options and complexity In-Reply-To: <20200312030917.GJ89512@eureka.lemis.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> <20200312030917.GJ89512@eureka.lemis.com> Message-ID: On Thu, 12 Mar 2020, Greg 'groggy' Lehey wrote: > On Wednesday, 11 March 2020 at 20:53:12 -0400, Steve Nickolas wrote: >> I went through all the switches defined by POSIX, and figured that >> those 26 could be cut down. > > A brave man to defy POSIX! I wasn't so brave, which is why we have > the -y option. xD >> My concept reduced the number of switches from 26 to 9 (FLRadfiln). >> Of course, the idea is to be more minimalist than POSIX, so some >> people's opinions on what is or isn't necessary may differ from >> mine. > > OK, let's compare notes: > >> I felt -A was a redundant "almost -a". > > Arguably -a could go too. The distinction seems arbitrary. Well, I think one or the other would be desirable. I figured -a was the better to keep - since it shows all dotfiles where -A leaves off . and .. . >> I felt -C and -x were redundant because a tool like column(1) could be >> used to do the same job (even though column(1) isn't POSIX). > > Neither would this ls(1) be. Of course. ;) > -S isn't POSIX. And to implement it without an option would mean > removing -h. -h is a gnuism, isn't it? https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html does specify the -S switch. That's POSIX, isn't it? > As I mentioned earlier, -t can't be done by a filter without > significantly modifying the timestamp output. That was my rationale > for the -D option, which allows sorting by an external filter. Understandable. Honestly if the date format weren't standardized as it were, I would've standardized on "yyyy-mm-dd,mm:ss" - which wouldn't need special processing in order to pump into sort(1). >> I felt -c and -u were meaningless, but that's because of the filesystems I >> usually work with that do not have functional equivalents. -u for one is >> completely useless on VFAT even though it has such timestamps! YMMV. > > I think this says more about your file systems than about the options. > I find both incredibly useful, and there's no easy way to get the > information elsewhere. stat(1) would be an option, but then that > could replace ls(1) completely. Perhaps true. > So, any others? > > -G: Colorized output. I'd be *really* happy to get rid of this, but > it's not easy to instate with a filter, so I suppose there are > enough people who like it that it will have to stay. > > -P: Seems only to be there to cancel a -H or -L. > > -W: "Display whiteouts when scanning directories". I don't even > understand what that is. I was using the link I referenced as my "standard", which doesn't have any of those. I can take or leave color ls. I don't like the GNU defaults because dark blue is TOO dark on my default settings. I think the flags are adequate to know what kind of file I'm dealing with. > -f: We haven't really discussed this one. If you want to remove -S, > -r and -t, then arguably -f should become the default and be > -removed. I used to use "dir|sort" a lot on PC DOS before it got "dir /o" in 5.0. I wouldn't have a problem with removing sort from ls altogether. > Of course, none of this will happen. But it is interesting to think > about it. In particular, options like -g and -o, which are no longer > modern. -uso. From dave at horsfall.org Thu Mar 12 15:22:01 2020 From: dave at horsfall.org (Dave Horsfall) Date: Thu, 12 Mar 2020 16:22:01 +1100 (EST) Subject: [TUHS] Command line options and complexity In-Reply-To: <20200311225638.GG89512@eureka.lemis.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> Message-ID: On Thu, 12 Mar 2020, Greg 'groggy' Lehey wrote: > A good example. But you're not removing options, you're just redefining > them. In fact I find the -h option particularly emetic, so a better > choice in removing options would be to remove -h and use a filter to > mutilate the sizes: > > $ ls -l | humanize I also had something like that in mind, except being British/Australian I'd spell it with an "s" :-) > But that's a pain, isn't it? That's why there's a -h option for people > who like it. Note that you can't do it the other way round: you can't > get the exact size from -h output. Which is why I suggested there be a means to turn it off; I'm becoming a fan of environment variables to modify the standard behaviour of tools (but I loathe the Penguin/OS default to use colours). > And then there's the question why you don't like the standard output. > Because the number strings are too long and difficult to read, maybe? > That's the rationale for the -, option. More than likely; as I approach age 68 I notice that I'm losing some cognitive facility... I might start using "," and see if I like it, but I see that the Mac doesn't have it (my Penguin is off the air at the moment), and having it as an environment variable would be nice. >> Quickly now, without looking: which option shows unprintable >> characters in a filename? Unless you use it regularly (in which >> case you have real problems) you would have to look it up; I find >> that "ls ... | od -bc" to be quicker, especially on filenames with >> trailing blanks etc (which "-B" won't show). > > This is arguably a bug in the -B option. I certainly don't think the > pipe notation is quicker. But it's nice to have both alternatives. Agreed; as for the bug I think it comes down to what is meant by an unprintable character. I certainly remember finding "hidden" set-uid shells with the name of ".. " etc back when I was going after the UNSW kiddies with an axe back in the late 70s... -- Dave From usotsuki at buric.co Thu Mar 12 15:35:14 2020 From: usotsuki at buric.co (Steve Nickolas) Date: Thu, 12 Mar 2020 01:35:14 -0400 (EDT) Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> Message-ID: On Thu, 12 Mar 2020, Dave Horsfall wrote: > Which is why I suggested there be a means to turn it off; I'm becoming a fan > of environment variables to modify the standard behaviour of tools (but I > loathe the Penguin/OS default to use colours). When I first used Linux, that wasn't the default. Personally, I don't think it should be (actually I think there simply shouldn't be a color mode at all to ls). > More than likely; as I approach age 68 I notice that I'm losing some > cognitive facility... I might start using "," and see if I like it, but I > see that the Mac doesn't have it (my Penguin is off the air at the moment), > and having it as an environment variable would be nice. GNU ls does not appear to have a -, switch. IBM, interestingly, introduced an environment variable in PC DOS 6.3 that did the opposite thing. If the NO_SEP variable existed, it suppressed commas in file sizes. -uso. From dave at horsfall.org Thu Mar 12 15:38:44 2020 From: dave at horsfall.org (Dave Horsfall) Date: Thu, 12 Mar 2020 16:38:44 +1100 (EST) Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> Message-ID: On Wed, 11 Mar 2020, Steve Nickolas wrote: > I felt -c and -u were meaningless, but that's because of the filesystems > I usually work with that do not have functional equivalents. -u for one > is completely useless on VFAT even though it has such timestamps! > YMMV. I find those flags really useful when doing forensic analysis on a file system :-) One particular instance was at $ORKPLACE some years back when a critical chunk of a file system had somehow disappeared overnight (it was our source base!). I got to work by comparing login sessions with those someone-unknown "ls" flags and had just about nailed the perp who was online at the time when I was ordered off it in no uncertain terms. Ummm, did I mention that my then $BOSS had a habit of working from home after a few (and quite a few) drinks? As I said, I was this -><- far away from fingering him... As it stood I knew who it was but wasn't able to prove it in time. -- Dave From peter at rulingia.com Thu Mar 12 16:48:07 2020 From: peter at rulingia.com (Peter Jeremy) Date: Thu, 12 Mar 2020 17:48:07 +1100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> Message-ID: <20200312064807.GA68127@server.rulingia.com> On 2020-Mar-11 20:53:12 -0400, Steve Nickolas wrote: >On Thu, 12 Mar 2020, Greg 'groggy' Lehey wrote: >> a better choice in removing options would be to remove -h and use a >> filter to mutilate the sizes: >> >> $ ls -l | humanize How does humanize decide which column to work on? If it only works on "ls -l", then it's not useful if I want other columns as well. Maybe it could just humanize any large number it found, but you probably don't want to "humanize" the inode number or filename. >I felt -s was a redundant "kindasorta -l". Except they are reporting completely different things - consider sparse files or filesystems (like ZFS) that support compression. -- Peter Jeremy -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From usotsuki at buric.co Thu Mar 12 17:37:03 2020 From: usotsuki at buric.co (Steve Nickolas) Date: Thu, 12 Mar 2020 03:37:03 -0400 (EDT) Subject: [TUHS] Command line options and complexity In-Reply-To: <20200312064807.GA68127@server.rulingia.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> <20200312064807.GA68127@server.rulingia.com> Message-ID: On Thu, 12 Mar 2020, Peter Jeremy wrote: > On 2020-Mar-11 20:53:12 -0400, Steve Nickolas wrote: > >> I felt -s was a redundant "kindasorta -l". > > Except they are reporting completely different things - consider sparse > files or filesystems (like ZFS) that support compression. I was under the impression that -s simply showed the file size divided by 512 and didn't account for sparseness or compression. (Of the filesystems I frequently work with, one of them does actually support sparseness (ProDOS).) -uso. From imp at bsdimp.com Thu Mar 12 17:42:33 2020 From: imp at bsdimp.com (Warner Losh) Date: Thu, 12 Mar 2020 01:42:33 -0600 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> <20200312064807.GA68127@server.rulingia.com> Message-ID: On Thu, Mar 12, 2020, 1:37 AM Steve Nickolas wrote: > On Thu, 12 Mar 2020, Peter Jeremy wrote: > > > On 2020-Mar-11 20:53:12 -0400, Steve Nickolas wrote: > > > >> I felt -s was a redundant "kindasorta -l". > > > > Except they are reporting completely different things - consider sparse > > files or filesystems (like ZFS) that support compression. > > I was under the impression that -s simply showed the file size divided by > 512 and didn't account for sparseness or compression. > Stat returns two values. The offset of the last byte and the number of blocks allocated to the file. Useful if you have a sparse file too... Warner (Of the filesystems I frequently work with, one of them does actually > support sparseness (ProDOS).) > > -uso. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jpl.jpl at gmail.com Thu Mar 12 22:57:09 2020 From: jpl.jpl at gmail.com (John P. Linderman) Date: Thu, 12 Mar 2020 08:57:09 -0400 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200305020517.GA13872@wopr> <20200308052632.GD20478@eureka.lemis.com> <202003080532.0285WcWn1544496@darkstar.fourwinds.com> <20200309212257.GB39634@wopr> Message-ID: My error. I was looking at getopt(1) rather than getopt(3). Of course optind is documented, it's the way to find non-flag arguments. I don't know why the Hancock authors chose to make rsort into a subroutine rather than just piping into the command. Perhaps something to do with the software release process? On Wed, Mar 11, 2020 at 5:29 PM Warner Losh wrote: > > > On Wed, Mar 11, 2020 at 11:43 AM John P. Linderman > wrote: > >> This is *great*, Kurt. The source in src/runtime/hrs/src for rsort.c is >> their version of my external sort, modified to be a subroutine. There's >> some lessons to be learned about "software hygiene". I was cavalier about >> freeing what I allocated dynamically. As a result, their version leaks like >> a sieve if the subroutine is called repeatedly. Apropos of which, they came >> to me having noted that only the first call was acting as expected. There's >> a wonderful irony (I'm big on irony). I had replaced my do-it-yourself >> argument processing with getopt. The code has the following comment >> >> ** Use getopt() for portability. >> >> A few lines later, you see >> >> optind = 1; /* reset after use in Hancock program * >> while ((c = getopt(argc, argv, "cCiIjmrsSuvb:f:D:o:p:T:x:y:z:")) != >> EOF) { >> >> optind??? Seems getopt has an undocumented global flag to prevent >> reprocessing the arguments. How portable:-) >> > > It's documented: > > The variables opterr and optind are both initialized to 1. The optind > variable may be set to another value before a set of calls to > getopt() in > order to skip over more or less argv entries. > > is what the FreeBSD man page has to say about it. So this just resets any > scanning that had happened before this... > > Warner > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steffen at sdaoden.eu Fri Mar 13 05:24:38 2020 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Thu, 12 Mar 2020 20:24:38 +0100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200305020517.GA13872@wopr> <20200308052632.GD20478@eureka.lemis.com> <202003080532.0285WcWn1544496@darkstar.fourwinds.com> <20200309212257.GB39634@wopr> Message-ID: <20200312192438.q9Q4k%steffen@sdaoden.eu> John P. Linderman wrote in : |My error. I was looking at getopt(1) rather than getopt(3). Of course \ |optind is documented, it's the way to find non-flag arguments. |I don't know why the Hancock authors chose to make rsort into a subroutine \ |rather than just piping into the command. Perhaps something to do with \ |the software release process? I really like a lot of such old code, and reading it. One can only learn from it. Even though i discovered all this in (Free)BSD land, after coming over from Linux, I loved reading those "old-hand" comment blocks, it was inspiration and kindled something here. For the few pieces of code that i am prowd of aka that i thought were worth it i followed their example. This rsort.c is however more verbose and spiritful than anything i ever wrote. I keep it in my box of precious things. getopt(3) on the other hand is portable but terrible. Just on the 10th i resorted a small SCSI MMC-3 cdda access tool (~50 KB C source are necessary for that in 2020, missing Solaris and MacOS, but including CD-TEXT and all that!!) to it because people are used to option and/or argument joining etc, but it lost long option support. Not worth commenting a lot, but here is an option parser of 6359 bytes when development verification code and dump_doc() are not counted, but is uses a carrier struct, supports long options, and documentation strings as part of long option strings (one .RODATA entry). FreeBSD's standard compatible and thus naked lib/libc/stdlib/getopt.c is 4312 bytes. And GNU's getopt_long is huge and even permutates arguments. At least getopt(3) is predictable once a user gets it. Things are different for sed(1)s -i and some sccs commands i have forgotten. I think it has even be tried to standardize optional arguments in that respect, but i would argue this is not a good direction to go, consider for example "sed -ie". Isn't this asking for troubles without accompanying comments. * static char const a_sopts[] = "A:h#"; * static char const * const a_lopts[] = { * "account:;A;" N_("execute an `account' command"), .. * "long-help;\201;" N_("this listing"), * NIL * }; .. * struct su_avopt avo; .. * su_avopt_setup(&avo, --argc, C(char const*const*,++argv), * a_sopts, a_lopts); * while((i = su_avopt_parse(&avo)) != su_AVOPT_STATE_DONE){ * switch(i){ * case 'A': * "account_name" = avo.avo_current_arg; * break; * case 'h': * case S(char,S(u8,'\201')): * a_main_usage(n_stdout); * if(i != 'h'){ * fprintf(n_stdout, "\nLong options:\n"); * su_avopt_dump_doc(&avo, &a_main_dump_doc, S(up,n_stdout)); * } * exit(0); .. * argc = avo.avo_argc; * argv = C(char**,avo.avo_argv); --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From grog at lemis.com Fri Mar 13 09:57:35 2020 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Fri, 13 Mar 2020 10:57:35 +1100 Subject: [TUHS] Command line options and complexity In-Reply-To: <20200312064807.GA68127@server.rulingia.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> <20200312064807.GA68127@server.rulingia.com> Message-ID: <20200312235735.GL89512@eureka.lemis.com> On Thursday, 12 March 2020 at 17:48:07 +1100, Peter Jeremy wrote: > On 2020-Mar-11 20:53:12 -0400, Steve Nickolas wrote: >> On Thu, 12 Mar 2020, Greg 'groggy' Lehey wrote: >>> a better choice in removing options would be to remove -h and use a >>> filter to mutilate the sizes: >>> >>> $ ls -l | humanize > > How does humanize decide which column to work on? It knows. It was written that way. > If it only works on "ls -l", then it's not useful if I want other > columns as well. Right. You'd have to change it. Recall that this was just an example. > Maybe it could just humanize any large number it found, but you > probably don't want to "humanize" the inode number or filename. Yes, this is exactly the scenario I described in an earlier mail message, where I called it $ ls -l | commafy 5 Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From grog at lemis.com Fri Mar 13 10:36:47 2020 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Fri, 13 Mar 2020 11:36:47 +1100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> Message-ID: <20200313003647.GM89512@eureka.lemis.com> On Thursday, 12 March 2020 at 16:22:01 +1100, Dave Horsfall wrote: > On Thu, 12 Mar 2020, Greg 'groggy' Lehey wrote: > >> A good example. But you're not removing options, you're just redefining >> them. In fact I find the -h option particularly emetic, so a better >> choice in removing options would be to remove -h and use a filter to >> mutilate the sizes: >> >> $ ls -l | humanize > > I also had something like that in mind, except being British/Australian > I'd spell it with an "s" :-) It's a common misconception that -ize is US English. The Oxford English Dictionary, normally not prescriptive, prefers it. See https://www.oed.com/page/faqs/Frequently+asked+questions#spell. I personally had -ise drummed out of me by my uncle, very much Australian. >> And then there's the question why you don't like the standard output. >> Because the number strings are too long and difficult to read, maybe? >> That's the rationale for the -, option. > > More than likely; as I approach age 68 I notice that I'm losing some > cognitive facility... I might start using "," and see if I like it, but I > see that the Mac doesn't have it (my Penguin is off the air at the > moment), and having it as an environment variable would be nice. Yes, currently only FreeBSD has it. But you have the sources. Apart from option handling, it's only: --- print.c (.../head/bin/ls/print.c) (revision 241014) +++ print.c (.../stable/10/bin/ls/print.c) (working copy) @@ -606,6 +606,10 @@ humanize_number(buf, sizeof(buf), (int64_t)bytes, "", HN_AUTOSCALE, HN_B | HN_NOSPACE | HN_DECIMAL); (void)printf("%*s ", (u_int)width, buf); + } else if (f_thousands) { /* with commas */ + /* This format assignment needed to work round gcc bug. */ + const char *format = "%*j'd "; + (void)printf(format, (u_int)width, bytes); } else (void)printf("%*jd ", (u_int)width, bytes); } A quick and dirty fix would be simply to replace the format string. Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From grog at lemis.com Fri Mar 13 11:02:06 2020 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Fri, 13 Mar 2020 12:02:06 +1100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> <20200312030917.GJ89512@eureka.lemis.com> Message-ID: <20200313010206.GN89512@eureka.lemis.com> On Wednesday, 11 March 2020 at 23:34:46 -0400, Steve Nickolas wrote: > On Thu, 12 Mar 2020, Greg 'groggy' Lehey wrote: >> -S isn't POSIX. And to implement it without an option would mean >> removing -h. > > -h is a gnuism, isn't it? It might have originated there, but then I would expect it to be spelt '--produce-human-readable-output'. I haven't been able to establish from the FreeBSD sources or commit logs when it was introduced. It would clearly have been a reimplementation. > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html does > specify the -S switch. That's POSIX, isn't it? So it is! This was the first option that I wanted to add, back when I still had practice wheels. I asked my mentor, and he said "not the Unix way", so I let it be. Then Wes Peters came up with the idea, and I thought he committed it, but it seems that it ultimately came from Kostas Blekos in 2005, based on the same feature on NetBSD and OpenBSD. I wonder when it made it to POSIX. >> As I mentioned earlier, -t can't be done by a filter without >> significantly modifying the timestamp output. That was my rationale >> for the -D option, which allows sorting by an external filter. > > Understandable. > > Honestly if the date format weren't standardized as it were, I would've > standardized on "yyyy-mm-dd,mm:ss" - which wouldn't need special > processing in order to pump into sort(1). Yes, that was one of the possibilities I thought of. Another obvious one was time_t, which is even easier to process. And then there's ISO 8601. That's why it didn't take me long to decide "do it *your* way” with the -D option. Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From doug at cs.dartmouth.edu Fri Mar 13 13:04:03 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Thu, 12 Mar 2020 23:04:03 -0400 Subject: [TUHS] Command options and complexity Message-ID: <202003130304.02D343ok099975@tahoe.cs.Dartmouth.EDU> > -,: Make the option standard: output numbers with commas every 3 digits A terrible idea. Whatever ls outputs should be easy for other programs to read, and few know how to read commafied numbers. As others have mentioned, this is also a strong argument for changing the output representation of dates. I often do mailx -H | sort -t/ -k2nr to sort in reverse order of size--a quick way to find the pay dirt when I want to shrink my mailbox. This would never fly if the sizes had commas. (Well, I suppose I could add sed s/,//g to the pipeline.) Doug From grog at lemis.com Fri Mar 13 14:25:27 2020 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Fri, 13 Mar 2020 15:25:27 +1100 Subject: [TUHS] Command options and complexity In-Reply-To: <202003130304.02D343ok099975@tahoe.cs.Dartmouth.EDU> References: <202003130304.02D343ok099975@tahoe.cs.Dartmouth.EDU> Message-ID: <20200313042527.GA44529@eureka.lemis.com> On Thursday, 12 March 2020 at 23:04:03 -0400, Doug McIlroy wrote: >> -,: Make the option standard: output numbers with commas every 3 digits > > A terrible idea. Yes, of course. I don't think any of us meant these ideas seriously. Was this the only objection you had? > I often do mailx -H | sort -t/ -k2nr to sort in reverse order of size--a > quick way to find the pay dirt when I want to shrink my mailbox. > This would never fly if the sizes had commas. (Well, I suppose I > could add sed s/,//g to the pipeline.) In fact, at least FreeBSD sort -n handles the numbers quite happily. I've tried it with 4.4BSD sort, which with a little coaxing builds under FreeBSD. It doesn't: it apparently sees the , as a delimiter. The real question is whether a comma is a valid character in an integer. There are arguments that it should be. Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From dave at horsfall.org Fri Mar 13 20:45:21 2020 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 13 Mar 2020 21:45:21 +1100 (EST) Subject: [TUHS] Command line options and complexity Message-ID: Meant for the list (and don't get me started on Reply All)... -- Dave ---------- Forwarded message ---------- Date: Fri, 13 Mar 2020 21:43:51 +1100 (EST) From: Dave Horsfall To: Greg 'groggy' Lehey Subject: Re: [TUHS] Command line options and complexity On Fri, 13 Mar 2020, Greg 'groggy' Lehey wrote: >> -h is a gnuism, isn't it? > > It might have originated there, but then I would expect it to be spelt > '--produce-human-readable-output'. I haven't been able to establish from the > FreeBSD sources or commit logs when it was introduced. It would clearly have > been a reimplementation. It's in "df" as well, praise Cthulu: aneurin# df -h Filesystem Size Used Avail Capacity Mounted on /dev/ad0s1a 496M 302M 154M 66% / devfs 1.0K 1.0K 0B 100% /dev tmpfs 1000 272K 999M 0% /tmp /dev/ad0s1d 2.9G 1.4G 1.2G 54% /usr /dev/ad0s1e 989M 581M 329M 64% /var /dev/ad0s1f 3.9G 2.2G 1.4G 62% /home /dev/ad0s1g 8.9G 8.0G 127M 98% /usr/local fdescfs 1.0K 1.0K 0B 100% /dev/fd procfs 4.0K 4.0K 0B 100% /proc (Memo to self: see where all the room has gone in /usr/local, as that's where I assigned the leftover space after the other partitions.) No, I've never liked stuffing everything under the root file system as both the Mac and Penguin do; fill the root file system and you're hosed (and I also have an itch about /tmp being there as it's a world-writable directory). >> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html does >> specify the -S switch. That's POSIX, isn't it? > > So it is! This was the first option that I wanted to add, back when I still > had practice wheels. I asked my mentor, and he said "not the Unix way", so I > let it be. Then Wes Peters came up with the idea, and I thought he committed > it, but it seems that it ultimately came from Kostas Blekos in 2005, based on > the same feature on NetBSD and OpenBSD. I wonder when it made it to POSIX. Years ago I wrote a simple script "lss" which did the sort after being howled down on one of the FreeBSD lists; what a surprise to see "-S"... Heck, back in my UNSW days I suggested extending stty() to cover non-TTY devices and got trashed by the AGSM/ElecEng mob; well well, look at ioctl() when it appeared. -- Dave From dave at horsfall.org Fri Mar 13 21:26:45 2020 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 13 Mar 2020 22:26:45 +1100 (EST) Subject: [TUHS] Command line options and complexity In-Reply-To: <20200313003647.GM89512@eureka.lemis.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> <20200313003647.GM89512@eureka.lemis.com> Message-ID: On Fri, 13 Mar 2020, Greg 'groggy' Lehey wrote: >>> $ ls -l | humanize >> >> I also had something like that in mind, except being British/Australian >> I'd spell it with an "s" :-) > > It's a common misconception that -ize is US English. The Oxford English > Dictionary, normally not prescriptive, prefers it. See > https://www.oed.com/page/faqs/Frequently+asked+questions#spell. I > personally had -ise drummed out of me by my uncle, very much Australian. I'm familiar with that (and also the fact that "aluminum" and "color" etc were British spelling). Being born and bred British with pedantic parents I've always hated "American" spelling as we called it, and it's sad to see such noted media as the Sydney Morning Herald slowly adopting it over the past few years; Australia has used British spelling at least since I emigrated here in 1965. Oh, it was meant to be a creat/create joke, BTW... >> More than likely; as I approach age 68 I notice that I'm losing some >> cognitive facility... I might start using "," and see if I like it, >> but I see that the Mac doesn't have it (my Penguin is off the air at >> the moment), and having it as an environment variable would be nice. > > Yes, currently only FreeBSD has it. But you have the sources. Apart > from option handling, it's only: [...] I don't like my chances with suggesting that to Apple; I'm not even sure if they even take user contributions (although back when I was on the dole and having delusions of grandeur I did register as an Apple developer, but I suspect that that's for non-Apple stuff i.e. it goes into the Apple Store). > A quick and dirty fix would be simply to replace the format string. I have done the odd binary patch (usually to reconfigure Unify database volumes back when I was with FGH)... Not right now, though, as it's time for bed. -- Dave From steffen at sdaoden.eu Sat Mar 14 05:48:18 2020 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Fri, 13 Mar 2020 20:48:18 +0100 Subject: [TUHS] Command options and complexity In-Reply-To: <202003130304.02D343ok099975@tahoe.cs.Dartmouth.EDU> References: <202003130304.02D343ok099975@tahoe.cs.Dartmouth.EDU> Message-ID: <20200313194818.XS-yZ%steffen@sdaoden.eu> Doug McIlroy wrote in <202003130304.02D343ok099975 at tahoe.cs.Dartmouth.EDU>: |> -,: Make the option standard: output numbers with commas every 3 digits | |A terrible idea. Whatever ls outputs should be easy for other |programs to read, and few know how to read commafied numbers. |As others have mentioned, this is also a strong argument for |changing the output representation of dates. | |I often do mailx -H | sort -t/ -k2nr to sort in reverse order of size--a |quick way to find the pay dirt when I want to shrink my mailbox. |This would never fly if the sizes had commas. (Well, I suppose I |could add sed s/,//g to the pipeline.) It was not fully developed code why you need to do this with the old mailx you use. With v14.9.11 that is in Debian stable (it is called s-nail there) and Ubuntu since "Disco Dingo" (i do not know Ubuntu, but i seem to recall you said you were using it) that becomes mailx -Sautosort=date -H [| tac] for example. But you could also use -L and do something like mailx -H -L '(larger 5000)' # > 5000 bytes mailx -H -L '(before 1-Mar-2020)' mailx -H -L '(since 1-Mar-2020)' or even go a batch approach (`search' == `from'): printf 'search "(before 1-Mar-2020)" # sh(1)ell quoting needed! move ` +elder # move the last selection to $folder/elder xit ' | mailx -Squiet -Snoheader This can be improved further (-# batch mode, for example). A lot of things are missing yet, however, these examples use IMAP search expression syntax, a compile-time option. No "regular" approach to these things yet. Anyway, that mailx is 4670 changesets ahead of what you use. It is unfortunately not the place, date and time to grouch about the Debian punishment of small projects with fewest developers, the latest release is 5554 ahead (and enters unstable; in testing there is one that is 5204 changesets ahead). These are _my_ changesets: with a Dr. Hipp Fossil versioning philosophie the numbers could easily be doubled or tripled. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From doug at cs.dartmouth.edu Sat Mar 14 09:31:36 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Fri, 13 Mar 2020 19:31:36 -0400 Subject: [TUHS] The most surprising Unix programs Message-ID: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> Once in a while a new program really surprises me. Reminiscing a while ago, I came up with a list of eye-opening Unix gems. Only a couple of these programs are indispensable or much used. What singles them out is their originality. I cannot imagine myself inventing any of them. What programs have struck you similarly? PDP-7 Unix The simplicity and power of the system caused me to turn away from big iron to a tiny machine. It offered the essence of the hierarchical file system, separate shell, and user-level process control that Multics had yet to deliver after hundreds of man-years' effort. Unix's lacks (e.g. record structure in the file system) were as enlightening and liberating as its novelties (e.g. shell redirection operators). dc The math library for Bob Morris's variable-precision desk calculator used backward error analysis to determine the precision necessary at each step to attain the user-specified precision of the result. In my software-components talk at the 1968 NATO conference on software engineering, I posited measurement-standard routines, which could deliver results of any desired precision, but did not know how to design one. dc still has the only such routines I know of. typo Typo ordered the words of a text by their similarity to the rest of the text. Typographic errors like "hte" tended to the front (dissimilar) end of the list. Bob Morris proudly said it would work as well on Urdu as it did on English. Although typo didn't help with phonetic misspellings, it was a godsend for amateur typists, and got plenty of use until the advent of a much less interesting, but more precise, dictionary-based spelling checker. Typo was as surprising inside as it was outside. Its similarity measure was based on trigram frequencies, which it counted in a 26x26x26 array. The small memory, which had barely room enough for 1-byte counters, spurred a scheme for squeezing large numbers into small counters. To avoid overflow, counters were updated probabilistically to maintain an estimate of the logarithm of the count. eqn With the advent of phototypesetting, it became possible, but hideously tedious, to output classical math notation. Lorinda Cherry set out to devise a higher-level description language and was soon joined by Brian Kernighan. Their brilliant stroke was to adapt oral tradition into written expression, so eqn was remarkably easy to learn. The first of its kind, eqn has barely been improved upon since. struct Brenda Baker undertook her Fortan-to-Ratfor converter against the advice of her department head--me. I thought it would likely produce an ad hoc reordering of the orginal, freed of statement numbers, but otherwise no more readable than a properly indented Fortran program. Brenda proved me wrong. She discovered that every Fortran program has a canonically structured form. Programmers preferred the canonicalized form to what they had originally written. pascal The syntax diagnostics from the compiler made by Sue Graham's group at Berkeley were the mmost helpful I have ever seen--and they were generated automatically. At a syntax error the compiler would suggest a token that could be inserted that would allow parsing to proceed further. No attempt was made to explain what was wrong. The compiler taught me Pascal in an evening, with no manual at hand. parts Hidden inside WWB (writer's workbench), Lorinda Cherry's Parts annotated English text with parts of speech, based on only a smidgen of English vocabulary, orthography, and grammar. From Parts markup, WWB inferred stylometrics such as the prevalance of adjectives, subordinate clauses, and compound sentences. The Today show picked up on WWB and interviewed Lorinda about it in the first TV exposure of anything Unix. egrep Al Aho expected his deterministic regular-expression recognizer would beat Ken's classic nondeterministic recognizer. Unfortunately, for single-shot use on complex regular expressions, Ken's could finish while egrep was still busy building a deterministic automaton. To finally gain the prize, Al sidestepped the curse of the automaton's exponentially big state table by inventing a way to build on the fly only the table entries that are actually visited during recognition. crabs Luca Cardelli's charming meta-program for the Blit window system released crabs that wandered around in empty screen space nibbling away at the ever more ragged edges of active windows. Some common threads Theory, though invisible on the surface, played a crucial role in the majority of these programs: typo, dc, struct, pascal, egrep. In fact much of their surprise lay in the novelty of the application of theory. Originators of nearly half the list--pascal, struct, parts, eqn--were women, well beyond women's demographic share of computer science. Doug McIlroy March, 2020 From dave at horsfall.org Sat Mar 14 10:40:45 2020 From: dave at horsfall.org (Dave Horsfall) Date: Sat, 14 Mar 2020 11:40:45 +1100 (EST) Subject: [TUHS] The most surprising Unix programs In-Reply-To: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> Message-ID: On Fri, 13 Mar 2020, Doug McIlroy wrote: [ List elided ] For me coming from RSX-11D and RSTS/E it would have to be the shell; I never realised that the command interpreter could be divorced from the OS, along with the concept of building new tools with pipes. Don't like how the shell works? Then just write your own, as there was nothing special about it... > Originators of nearly half the list--pascal, struct, parts, eqn--were > women, well beyond women's demographic share of computer science. I never realised that; it is indeed worth remembering. -- Dave From woods at robohack.ca Sat Mar 14 12:13:53 2020 From: woods at robohack.ca (Greg A. Woods) Date: Fri, 13 Mar 2020 19:13:53 -0700 Subject: [TUHS] Command line options and complexity In-Reply-To: <20200313003647.GM89512@eureka.lemis.com> References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> <20200313003647.GM89512@eureka.lemis.com> Message-ID: At Fri, 13 Mar 2020 11:36:47 +1100, Greg 'groggy' Lehey wrote: Subject: Re: [TUHS] Command line options and complexity > > On Thursday, 12 March 2020 at 16:22:01 +1100, Dave Horsfall wrote: > > On Thu, 12 Mar 2020, Greg 'groggy' Lehey wrote: > > > > > > And then there's the question why you don't like the standard output. > > > Because the number strings are too long and difficult to read, maybe? > > > That's the rationale for the -, option. > > > > More than likely; as I approach age 68 I notice that I'm losing some > > cognitive facility... I might start using "," and see if I like it, but I > > see that the Mac doesn't have it (my Penguin is off the air at the > > moment), and having it as an environment variable would be nice. > > Yes, currently only FreeBSD has it. Because of course NetBSD has chosen a different option letter: 'M' Unfortunately on NetBSD and FreeBSD the appearance of commas (or whatever is appropriate) depends on the locale being correctly configured, and this is not always so easy to do! -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From woods at robohack.ca Sat Mar 14 12:24:20 2020 From: woods at robohack.ca (Greg A. Woods) Date: Fri, 13 Mar 2020 19:24:20 -0700 Subject: [TUHS] Command options and complexity In-Reply-To: <20200313042527.GA44529@eureka.lemis.com> References: <202003130304.02D343ok099975@tahoe.cs.Dartmouth.EDU> <20200313042527.GA44529@eureka.lemis.com> Message-ID: At Fri, 13 Mar 2020 15:25:27 +1100, Greg 'groggy' Lehey wrote: Subject: Re: [TUHS] Command options and complexity > > The real question is whether a comma is a valid character in an > integer. There are arguments that it should be. It would make sense if scanf() had the equivalent flag to printf() to make number printing and parsing at least potentially orthogonal. -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From grog at lemis.com Sat Mar 14 14:31:03 2020 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Sat, 14 Mar 2020 15:31:03 +1100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <20200311225638.GG89512@eureka.lemis.com> <20200313003647.GM89512@eureka.lemis.com> Message-ID: <20200314043103.GC44529@eureka.lemis.com> On Friday, 13 March 2020 at 19:13:53 -0700, Greg A. Woods wrote: > At Fri, 13 Mar 2020 11:36:47 +1100, Greg 'groggy' Lehey wrote: >> Yes, currently only FreeBSD has it. > > Because of course NetBSD has chosen a different option letter: 'M' Oh. Somehow I missed that. Damn. > Unfortunately on NetBSD and FreeBSD the appearance of commas (or > whatever is appropriate) depends on the locale being correctly > configured, and this is not always so easy to do! Agreed. I've been meaning to default to , if the locale doesn't specify a delimiter, but haven't got round to it. Give me a problem report (https://bugs.freebsd.org/bugzilla/) and I'll fix it. Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From grog at lemis.com Sat Mar 14 14:35:05 2020 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Sat, 14 Mar 2020 15:35:05 +1100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: Message-ID: <20200314043505.GD44529@eureka.lemis.com> On Friday, 13 March 2020 at 21:45:21 +1100, Dave Horsfall wrote: > On Fri, 13 Mar 2020, Greg 'groggy' Lehey wrote: > >>> -h is a gnuism, isn't it? >> >> It might have originated there, but then I would expect it to be spelt >> '--produce-human-readable-output'. I haven't been able to establish from the >> FreeBSD sources or commit logs when it was introduced. It would clearly have >> been a reimplementation. > > It's in "df" as well, praise Cthulu: > > aneurin# df -h > Filesystem Size Used Avail Capacity Mounted on > /dev/ad0s1a 496M 302M 154M 66% / > /dev/ad0s1d 2.9G 1.4G 1.2G 54% /usr > /dev/ad0s1e 989M 581M 329M 64% /var ... It also has the , option: === grog at eureka (/dev/pts/72) ~ 8 -> df -, Filesystem 1048576-blocks Used Avail Capacity Mounted on /dev/ada0p4 39,662 21,918 14,571 60% / /dev/ada0p2 39,662 13,447 23,042 37% /destdir /dev/ada0p5 3,705,520 1,831,345 1,577,733 54% /home /dev/ada1p1 7,629,565 6,358,607 1,194,661 84% /Photos I find it much easier to see the relative size like that. Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From skogtun at gmail.com Sat Mar 14 21:30:33 2020 From: skogtun at gmail.com (Harald Arnesen) Date: Sat, 14 Mar 2020 12:30:33 +0100 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> Message-ID: <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> Doug McIlroy [14.03.2020 00:31]: > struct > > Brenda Baker undertook her Fortan-to-Ratfor converter against the advice > of her department head--me. I thought it would likely produce an ad hoc > reordering of the orginal, freed of statement numbers, but otherwise no > more readable than a properly indented Fortran program. Brenda proved > me wrong. She discovered that every Fortran program has a canonically > structured form. Programmers preferred the canonicalized form to what > they had originally written. I've been trying to find the source to this. I have lots of old spaghetti-Fortran routines from a former job, and I would love to try making it somewhat understandable. -- Hilsen Harald From clemc at ccc.com Sat Mar 14 22:24:04 2020 From: clemc at ccc.com (Clem Cole) Date: Sat, 14 Mar 2020 08:24:04 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> Message-ID: It's in the V7 sources and binaries in the TUHS archives: % find . -name '*struct*' -print ./bin/struct ./usr/lib/struct ./usr/lib/struct/structure ./usr/man/man1/struct.1 ./usr/src/cmd/struct On Sat, Mar 14, 2020 at 7:31 AM Harald Arnesen wrote: > Doug McIlroy [14.03.2020 00:31]: > > > struct > > > > Brenda Baker undertook her Fortan-to-Ratfor converter against the advice > > of her department head--me. I thought it would likely produce an ad hoc > > reordering of the orginal, freed of statement numbers, but otherwise no > > more readable than a properly indented Fortran program. Brenda proved > > me wrong. She discovered that every Fortran program has a canonically > > structured form. Programmers preferred the canonicalized form to what > > they had originally written. > > I've been trying to find the source to this. I have lots of old > spaghetti-Fortran routines from a former job, and I would love to try > making it somewhat understandable. > -- > Hilsen Harald > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich.salz at gmail.com Sat Mar 14 23:08:19 2020 From: rich.salz at gmail.com (Richard Salz) Date: Sat, 14 Mar 2020 09:08:19 -0400 Subject: [TUHS] Command options and complexity In-Reply-To: References: <202003130304.02D343ok099975@tahoe.cs.Dartmouth.EDU> <20200313042527.GA44529@eureka.lemis.com> Message-ID: > It would make sense if scanf() had the equivalent flag to printf() to > make number printing and parsing at least potentially orthogonal. > That is not an easy thing to do because of local (er, locale:) conventions. See https://docs.oracle.com/cd/E19455-01/806-0169/overview-9/index.html for examples. Having a canonical input and output format, which is converted by filters just before/after the human sees the numbers, seems the only way to make this work globally. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jpl.jpl at gmail.com Sun Mar 15 05:52:08 2020 From: jpl.jpl at gmail.com (John P. Linderman) Date: Sat, 14 Mar 2020 15:52:08 -0400 Subject: [TUHS] Command line options and complexity In-Reply-To: <20200314043505.GD44529@eureka.lemis.com> References: <20200314043505.GD44529@eureka.lemis.com> Message-ID: Here's a command I wrote long ago using a different way to deal with options: *isee* Usage: isee format file ... Display specified inode information for files passed as arguments. Items of the form ``%X'' in format will be replaced for these X: dev inode ino mode nlink uid gid rdev size atime mtime ctime now filename Parenthesized printf-style format specifications can follow a % to override the default format for the various items. %filename is the name of the current file argument. %now is the time (in seconds) when the command started running. The other items are from the stat structure. Example: isee "%(40s)filename: %mtime %mode" /dev/null Show file modification time and mode of /dev/null inode is just a synonym for ino. Instead of a kazillion options, the %-stat-field items identify *what* you want to see and the printf-style formats identify *how* you want them shown. Someone in the Murray Hill library added strftime formats for date fields, a fine addition, in my view. Adding readable user and group names rather than numerical ids would be worth considering. *Maybe* having a "rwx"-style form for mode. Sorting can be done by piping the output through sort. Don't get hung up on shortcomings of the command, just consider how a few familiar concepts and pipes can be combined to provide a large number of options. On Sat, Mar 14, 2020 at 12:35 AM Greg 'groggy' Lehey wrote: > On Friday, 13 March 2020 at 21:45:21 +1100, Dave Horsfall wrote: > > On Fri, 13 Mar 2020, Greg 'groggy' Lehey wrote: > > > >>> -h is a gnuism, isn't it? > >> > >> It might have originated there, but then I would expect it to be spelt > >> '--produce-human-readable-output'. I haven't been able to establish > from the > >> FreeBSD sources or commit logs when it was introduced. It would > clearly have > >> been a reimplementation. > > > > It's in "df" as well, praise Cthulu: > > > > aneurin# df -h > > Filesystem Size Used Avail Capacity Mounted on > > /dev/ad0s1a 496M 302M 154M 66% / > > /dev/ad0s1d 2.9G 1.4G 1.2G 54% /usr > > /dev/ad0s1e 989M 581M 329M 64% /var > ... > > It also has the , option: > > === grog at eureka (/dev/pts/72) ~ 8 -> df -, > Filesystem 1048576-blocks Used Avail Capacity Mounted on > /dev/ada0p4 39,662 21,918 14,571 60% / > /dev/ada0p2 39,662 13,447 23,042 37% /destdir > /dev/ada0p5 3,705,520 1,831,345 1,577,733 54% /home > /dev/ada1p1 7,629,565 6,358,607 1,194,661 84% /Photos > > I find it much easier to see the relative size like that. > > Greg > -- > Sent from my desktop computer. > Finger grog at lemis.com for PGP public key. > See complete headers for address and phone numbers. > This message is digitally signed. If your Microsoft mail program > reports problems, please read http://lemis.com/broken-MUA > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steffen at sdaoden.eu Sun Mar 15 06:25:49 2020 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Sat, 14 Mar 2020 21:25:49 +0100 Subject: [TUHS] Command line options and complexity In-Reply-To: References: <20200314043505.GD44529@eureka.lemis.com> Message-ID: <20200314202549.I-Zpc%steffen@sdaoden.eu> John P. Linderman wrote in : |Here's a command I wrote long ago using a different way to deal with \ |options: | |  isee |Usage: isee format file ... |    Display specified inode information for files passed as arguments. |    Items of the form ``%X'' in format will be replaced for these X: |dev inode ino mode nlink uid gid rdev size atime |mtime ctime now filename |    Parenthesized printf-style format specifications can follow a % |    to override the default format for the various items. |    %filename is the name of the current file argument. |    %now is the time (in seconds) when the command started running. |    The other items are from the stat structure. | |    Example: isee "%(40s)filename: %mtime %mode" /dev/null |    Show file modification time and mode of /dev/null | |inode is just a synonym for ino. | |Instead of a kazillion options, the %-stat-field items identify what \ |you want to see and the printf-style formats identify how you want \ |them shown. Someone in the Murray Hill library added strftime |formats for date fields, a fine addition, in my view. Adding readable \ |user and group names rather than numerical ids would be worth considering. \ |Maybe having a "rwx"-style form for mode. Sorting can be |done by piping the output through sort. Don't get hung up on shortcomings \ |of the command, just consider how a few familiar concepts and pipes \ |can be combined to provide a large number of options. When i switched to FreeBSD around 2001, the handbook was on the CDs i had, and i stumbled upon a very impressive assembler example. It is still there[1], at least in parts(?). Coming from C64, then DOS/4DOS and <2 years Linux, aka kid games, grey-industry, MS and xeyes background, i read [1] https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/x86-fpu.html Personally, I like to keep it simple. Something either is a number, so I process it. Or it is not a number, so I discard it. I do not like the computer complaining about me typing in an extra character when it is obvious that it is an extra character. Duh! Plus, it allows me to break up the monotony of computing and type in a query instead of just a number: What is the best pinhole diameter for the focal length of 150? There is no reason for the computer to spit out a number of complaints: Syntax error: What Syntax error: is Syntax error: the Syntax error: best Et cetera, et cetera, et cetera. Secondly, I like the # character to denote the start of a comment which extends to the end of the line. This does not take too much effort to code, and lets me treat input files for my software as executable scripts. and it was like being warped from Chaplin's Modern Times to a rich man's California style living! And that in assembler!! % pinhole Computer, What size pinhole do I need for the focal length of 150? 150 490 306 362 2930 12 Hmmm... How about 160? 160 506 316 362 3125 12 Let's make it 155, please. 155 498 311 362 3027 12 Ah, let's try 157... 157 501 313 362 3066 12 156? 156 500 312 362 3047 12 That's it! Perfect! Thank you very much! ^D Nonetheless: i never managed to create Hippie-proof programs in real life. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From tuhs at keck.us Mon Mar 16 04:40:25 2020 From: tuhs at keck.us (Cornelius Keck) Date: Sun, 15 Mar 2020 13:40:25 -0500 (CDT) Subject: [TUHS] Unix on Zilog Z8000? In-Reply-To: <25E62EB5E090E7CB.f810e0c9-fa0e-409f-880d-5f8dc51ff5e7@mail.outlook.com> References: <20200121182045.GO26619@mcvoy.com> <25E62EB5E090E7CB.f810e0c9-fa0e-409f-880d-5f8dc51ff5e7@mail.outlook.com> Message-ID: That would be Cohere 0.7.3. Snappy system, loved it. Commodore Germany in Braunschweig got told to ditch the project after successfully presenting it at CeBit in Hannover. Many of these systems made it into hobbyists' hands. It's pretty much a version 7 lookalike, but internally different enough to encur AT&T's wrath. No network. Loved that thing. It's in storage in Germany. Gotta wonder if it still works. Other stuff I've brought back here did, so.... On Tue, 21 Jan 2020, Jason Stevens wrote: > Date: Tue, 21 Jan 2020 22:35:59 +0000 (UTC) > From: Jason Stevens > To: Henry Bent , Larry McVoy > Cc: The Unix Heritage Society mailing list > Subject: Re: [TUHS] Unix on Zilog Z8000? > > The only one I know is Coherent.  Disk images recently surfaced > > > > > https://www.autometer.de/unix4fun/coherent/ftp/distrib/Coherent-0.7/ > > > > > This is for the Commodore B 900 prototype. > > > > > http://www.floodgap.com/retrobits/ckb/secret/900.html > > > > > Get Outlook for Android > > > > > > > > On Wed, Jan 22, 2020 at 2:22 AM +0800, "Larry McVoy" wrote: > > > > > > > > > > > On Tue, Jan 21, 2020 at 01:15:53PM -0500, Henry Bent wrote: > > On Tue, 21 Jan 2020 at 12:53, Jon Forrest wrote: > > > > > There's been a lot of discussion about early Unix on Intel, National > > > Semi, Motorola, and Sparc processors. I don't recall if Unix ran on > > > the Z8000, and if not, why not. > > > > > > > A tiny bit of research would have answered this question for you: > > https://en.wikipedia.org/wiki/Zilog_Z8000#Z8000_CPU_based_systems > > Yeah, it ran on the 16 bit one but I looked and couldn't find if they > got Unix on the z80000 (which I suspect is what Jon meant). > > > > > > From robpike at gmail.com Mon Mar 16 08:01:46 2020 From: robpike at gmail.com (Rob Pike) Date: Mon, 16 Mar 2020 09:01:46 +1100 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> Message-ID: Ha! Someone on hacker news, that fount of profundity, posted the C code for typo and said that it would be a simple matter to update to modern Unix. I tried quite hard to do that years ago, and failed, which is why I wrote my own version in Go. The original is just barely above assembler and deeply dependent on CPU, operating system, compiler and even linker. Hell, evidence shows it couldn't even be ported to v7. It's hard for younger programmers to appreciate how different things once were. -rob From lm at mcvoy.com Mon Mar 16 08:14:53 2020 From: lm at mcvoy.com (Larry McVoy) Date: Sun, 15 Mar 2020 15:14:53 -0700 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> Message-ID: <20200315221453.GG26660@mcvoy.com> On Mon, Mar 16, 2020 at 09:01:46AM +1100, Rob Pike wrote: > Ha! Someone on hacker news, that fount of profundity, posted the C > code for typo and said that it would be a simple matter to update to > modern Unix. I used to hang out on hacker news, it was one of the few technical places left. I left after noticing the guys who run it were protecting a wack job who every single time something negative was posted about Round Up, he came up with all sorts of nonsense to defend Monsanto. Clearly a paid schill. My dad died at 75 from non-Hodgkins Lymphoma, he wasn't a drinker, wasn't a smoker, swam multiple times a week, no history of cancer in the family. He was convinced that it was Round Up that did him in, he was a Physics prof but he sort of spanned disciplines, he taught in the biology department, ecology, he was all over the place. I'd post my story, the schill would shit all over me, I'd complain to the powers that be, people who knew me would push back on him, I'm one of the few systems people hanging out there. It got to be too much, I haven't been on hacker news for several years. Even when I was there, the stuff that passed for hacker info was largely pathetic. The one exception I remember was the Netflix write up of how they filled two 100Gbit/sec ethernet pipes with 200,000 TCP connections. That was extremely impressive, I've worked on problems like that and I can assure you that it is trivial to fill those pipes with 2 connections, it's extremely hard to do so with 200,000. And they were coming off SSD, up to user space, encrypt in user space, then send it back down to the kernel out the pipes. Super, super impressive but that's the one really interesting thing I've seen there in 20 years. From clemc at ccc.com Mon Mar 16 08:30:20 2020 From: clemc at ccc.com (Clem Cole) Date: Sun, 15 Mar 2020 18:30:20 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> Message-ID: On Sun, Mar 15, 2020 at 6:01 PM Rob Pike wrote: > It's hard for younger programmers to appreciate how different things once > were. > Amen bro. But I would add how to do well with less. To me, the marvel of V6 was how complete it still is. Other than the lack of a networking stack, and multiple windows/screens, much of what I still do day to day as a programmer can be done with 6th edition. I'd miss Go and some of the original features of ANSI C, but the editor's we had got the job, there was even a video version I was happy with. I used troff to make slides instead of PPT. I guess I would miss a Chrome and Gmail, if I have to go back to MH/HM for email. FWIW: I just wrote a QUORA answer and referred folks to your and Brian's original book. Someone wanted to know as a beginner the 'best way to learn Ubuntu.' -------------- next part -------------- An HTML attachment was scrubbed... URL: From boomer3200 at gmail.com Mon Mar 16 08:56:03 2020 From: boomer3200 at gmail.com (Dan Plassche) Date: Sun, 15 Mar 2020 18:56:03 -0400 Subject: [TUHS] BSD Net/2 Updates from CSRG Message-ID: Hi, Would anyone know how the CSRG made available updates to the public distribution of Net/2 before releasing 4.4 BSD Lite? Net/2 was released on tape in July 1991 as represented in the TUHS archive. The first mention I see of the sources appearing online is old usenet posts announcing the bsd-sources mirror on uu.net starting in January 1992. However, the Net/2 derived versions of NetBSD (0.8-0.9) and FreeBSD (1.0-1.1.5.1) imported userland tools revised after both dates in early 1993, when there are also mentions of using 4.4 code in the commit logs (well before Lite 1 was released in March 1994). At this point, I'm wondering if the latest code was issued on tape at the time of a request for Net/2 sources and there were direct uploads from the CSRG to any online mirrors before the USL lawsuit? Thanks, Dan Plassche -- Dan Plassche From dave at horsfall.org Mon Mar 16 09:20:57 2020 From: dave at horsfall.org (Dave Horsfall) Date: Mon, 16 Mar 2020 10:20:57 +1100 (EST) Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> Message-ID: On Mon, 16 Mar 2020, Rob Pike wrote: > Ha! Someone on hacker news, that fount of profundity, posted the C code > for typo and said that it would be a simple matter to update to modern > Unix. [ And it isn't...] Got a pointer for it? Wherever it is, it's well hidden... I'd like to see some of that ancient C code; for example, my favourite is: int abort 4; ... abort(); Yes, really :-) -- Dave From imp at bsdimp.com Mon Mar 16 09:34:38 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 15 Mar 2020 17:34:38 -0600 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <20200315221453.GG26660@mcvoy.com> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> <20200315221453.GG26660@mcvoy.com> Message-ID: On Sun, Mar 15, 2020 at 4:15 PM Larry McVoy wrote: > It got to be too much, I haven't been on hacker news for several years. > Even when I was there, the stuff that passed for hacker info was largely > pathetic. The one exception I remember was the Netflix write up of how > they filled two 100Gbit/sec ethernet pipes with 200,000 TCP connections. > That was extremely impressive, I've worked on problems like that and I > can assure you that it is trivial to fill those pipes with 2 connections, > it's extremely hard to do so with 200,000. And they were coming off SSD, > up to user space, encrypt in user space, then send it back down to the > kernel out the pipes. Super, super impressive but that's the one really > interesting thing I've seen there in 20 years. > Well, to be fair, we did eliminate the trip up to userland and back down and did the bulk crypto in the kernel... We did do the session negotiation in userland though, but once that was done, the rest was done in-line in the kernel... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From wobblygong at gmail.com Mon Mar 16 10:28:26 2020 From: wobblygong at gmail.com (Wesley Parish) Date: Mon, 16 Mar 2020 13:28:26 +1300 Subject: [TUHS] Unix on Zilog Z8000? In-Reply-To: References: <20200121182045.GO26619@mcvoy.com> <25E62EB5E090E7CB.f810e0c9-fa0e-409f-880d-5f8dc51ff5e7@mail.outlook.com> Message-ID: Is it still in existence? In the form of floppies? Any source? Thanks Wesley Parish On 3/16/20, Cornelius Keck wrote: > That would be Cohere 0.7.3. Snappy system, loved it. Commodore Germany in > Braunschweig got told to ditch the project after successfully presenting > it at CeBit in Hannover. Many of these systems made it into hobbyists' > hands. It's pretty much a version 7 lookalike, but internally different > enough to encur AT&T's wrath. No network. Loved that thing. It's in > storage in Germany. Gotta wonder if it still works. Other stuff I've > brought back here did, so.... > > > > On Tue, 21 Jan 2020, Jason Stevens wrote: > >> Date: Tue, 21 Jan 2020 22:35:59 +0000 (UTC) >> From: Jason Stevens >> To: Henry Bent , Larry McVoy >> Cc: The Unix Heritage Society mailing list >> Subject: Re: [TUHS] Unix on Zilog Z8000? >> >> The only one I know is Coherent. Disk images recently surfaced >> >> >> >> >> https://www.autometer.de/unix4fun/coherent/ftp/distrib/Coherent-0.7/ >> >> >> >> >> This is for the Commodore B 900 prototype. >> >> >> >> >> http://www.floodgap.com/retrobits/ckb/secret/900.html >> >> >> >> >> Get Outlook for Android >> >> >> >> >> >> >> >> On Wed, Jan 22, 2020 at 2:22 AM +0800, "Larry McVoy" >> wrote: >> >> >> >> >> >> >> >> >> >> >> On Tue, Jan 21, 2020 at 01:15:53PM -0500, Henry Bent wrote: >> > On Tue, 21 Jan 2020 at 12:53, Jon Forrest wrote: >> > >> > > There's been a lot of discussion about early Unix on Intel, National >> > > Semi, Motorola, and Sparc processors. I don't recall if Unix ran on >> > > the Z8000, and if not, why not. >> > > >> > >> > A tiny bit of research would have answered this question for you: >> > https://en.wikipedia.org/wiki/Zilog_Z8000#Z8000_CPU_based_systems >> >> Yeah, it ran on the 16 bit one but I looked and couldn't find if they >> got Unix on the z80000 (which I suspect is what Jon meant). >> >> >> >> >> >> > From robpike at gmail.com Mon Mar 16 10:56:22 2020 From: robpike at gmail.com (Rob Pike) Date: Mon, 16 Mar 2020 11:56:22 +1100 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> Message-ID: There's a copy in my github repo for it, along w2006.txt and the salt file. https://github.com/robpike/typo/tree/master/unix -rob On Mon, Mar 16, 2020 at 10:21 AM Dave Horsfall wrote: > > On Mon, 16 Mar 2020, Rob Pike wrote: > > > Ha! Someone on hacker news, that fount of profundity, posted the C code > > for typo and said that it would be a simple matter to update to modern > > Unix. > > [ And it isn't...] > > Got a pointer for it? Wherever it is, it's well hidden... I'd like to > see some of that ancient C code; for example, my favourite is: > > int abort 4; > ... > abort(); > > Yes, really :-) > > -- Dave From ality at pbrane.org Mon Mar 16 12:45:42 2020 From: ality at pbrane.org (Anthony Martin) Date: Sun, 15 Mar 2020 19:45:42 -0700 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> <20200315221453.GG26660@mcvoy.com> Message-ID: <20200316023637.GA121220@alice> Warner Losh once said: > Well, to be fair, we did eliminate the trip up to userland and back down > and did the bulk crypto in the kernel... We did do the session > negotiation in userland though, but once that was done, the rest was done > in-line in the kernel... Plan 9 has had the SSL record layer and bulk crypto in the kernel for almost 25 years. Just sayin'. ☺ Cheers, Anthony From pnr at planet.nl Mon Mar 16 22:31:54 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Mon, 16 Mar 2020 13:31:54 +0100 Subject: [TUHS] Unix 8th / 9th edition IPC paper Message-ID: <98DC6210-A6ED-4E51-8EF8-085CB553D966@planet.nl> There is an interesting paper that Dennis Ritchie and Dave Presotto wrote, “Interprocess Communication in the Ninth Edition Unix System” (https://www.bell-labs.com/usr/dmr/www/ipcpaper.html ). This appears to be an update of a paper that they wrote in 1985, “Interprocess Communication in the Eighth Edition Unix System”. This earlier paper is hard to find. Anybody on this list who has it on hand? -------------- next part -------------- An HTML attachment was scrubbed... URL: From chet.ramey at case.edu Mon Mar 16 23:09:59 2020 From: chet.ramey at case.edu (Chet Ramey) Date: Mon, 16 Mar 2020 09:09:59 -0400 Subject: [TUHS] Unix 8th / 9th edition IPC paper In-Reply-To: <98DC6210-A6ED-4E51-8EF8-085CB553D966@planet.nl> References: <98DC6210-A6ED-4E51-8EF8-085CB553D966@planet.nl> Message-ID: On 3/16/20 8:31 AM, Paul Ruizendaal wrote: > There is an interesting paper that Dennis Ritchie and Dave Presotto wrote, > “Interprocess Communication in the Ninth Edition Unix System” > (https://www.bell-labs.com/usr/dmr/www/ipcpaper.html). > > This appears to be an update of a paper that they wrote in 1985, > “Interprocess Communication in the Eighth Edition Unix System”. This > earlier paper is hard to find. > > Anybody on this list who has it on hand? It's supposed to be in the proceedings of the June, 1985 Usenix conference. If anyone has a hard copy, and the full paper is in there, it should be scannable. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ From clemc at ccc.com Tue Mar 17 00:40:54 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 16 Mar 2020 10:40:54 -0400 Subject: [TUHS] Unix 8th / 9th edition IPC paper In-Reply-To: References: <98DC6210-A6ED-4E51-8EF8-085CB553D966@planet.nl> Message-ID: I sent it off list to Chet and Paul but it's too big for Warren's reflector. I'll work with him to get it separately. If anyone else wants it before, please let me know off list. Clem On Mon, Mar 16, 2020 at 10:37 AM Clem Cole wrote: > Here you go... > > Warren - Would please be able to add to the docs/paper directories... > USENIX Summer 1985 June 11-14 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dfawcus+lists-tuhs at employees.org Tue Mar 17 08:02:05 2020 From: dfawcus+lists-tuhs at employees.org (Derek Fawcus) Date: Mon, 16 Mar 2020 23:02:05 +0100 Subject: [TUHS] Unix on Zilog Z8000? In-Reply-To: References: <20200121182045.GO26619@mcvoy.com> <25E62EB5E090E7CB.f810e0c9-fa0e-409f-880d-5f8dc51ff5e7@mail.outlook.com> Message-ID: <20200316220205.GA45265@clarinet.employees.org> On Mon, Mar 16, 2020 at 01:28:26PM +1300, Wesley Parish wrote: > Is it still in existence? > In the form of floppies? https://www.autometer.de/unix4fun/coherent/ftp/distrib/Coherent-0.7/ Certainly disk2 looks like it has a filesystem on it, since there are blocks which appear to be 16 byte directory entries, presumably inode number and 14 char name. > Any source? I don't know about historical sources of the time. The availale Coherent source has a few files with z8001 in their name: $ gzip -dc ~/Downloads/mwc-COHERENT-Source.tgz|tar tvf - |fgrep -i z800 -rw-r--r-- 0 steve staff 6521 6 Feb 1991 romana/relic/d/lib/libc/sys/mch/z8001.tar.Z drwxr-xr-x 0 steve staff 0 14 Jun 2003 romana/relic/d/lib/libc/stdio/z8001/ -rw-r--r-- 0 steve staff 268 9 Nov 1984 romana/relic/d/lib/libc/stdio/z8001/fgetw.c -rw-r--r-- 0 steve staff 169 9 Nov 1984 romana/relic/d/lib/libc/stdio/z8001/fputw.c -rw-r--r-- 0 steve staff 340 9 Nov 1984 romana/relic/d/lib/libc/stdio/z8001/getw.c -rw-r--r-- 0 steve staff 231 9 Nov 1984 romana/relic/d/lib/libc/stdio/z8001/putw.c -rw-r--r-- 0 steve staff 4906 6 Feb 1991 romana/relic/d/lib/libc/gen/mch/z8001.tar.Z drwxr-xr-x 0 steve staff 0 14 Jun 2003 romana/relic/d/bin/diff/z8001/ -rw-r--r-- 0 steve staff 8445 2 Nov 1984 romana/relic/d/bin/diff/z8001/diff1.c -rw-r--r-- 0 steve staff 7767 23 Mar 1990 romana/relic/d/bin/diff/z8001/diff2.c -rw-r--r-- 0 steve staff 92 8 Jul 1985 romana/relic/d/bin/diff/z8001/Readme -rw-r--r-- 0 steve staff 17512 7 Mar 1991 romana/relic/d/bin/db/mch/z8001.tar.Z -rw-r--r-- 0 steve staff 13929 7 Mar 1991 romana/relic/d/bin/db/mch/z8002.tar.Z -rw-r--r-- 0 steve staff 21823 6 Feb 1991 romana/relic/d/bin/as/mch/z8001.tar.Z -rw-r--r-- 0 steve staff 19275 6 Feb 1991 romana/relic/d/bin/as/mch/z8002.tar.Z -rw-r--r-- 0 steve staff 6521 6 Feb 1991 romana/relic/b/lib/libc/sys/mch/z8001.tar.Z drwxr-xr-x 0 steve staff 0 14 Jun 2003 romana/relic/b/lib/libc/stdio/z8001/ -rw-r--r-- 0 steve staff 268 9 Nov 1984 romana/relic/b/lib/libc/stdio/z8001/fgetw.c -rw-r--r-- 0 steve staff 169 9 Nov 1984 romana/relic/b/lib/libc/stdio/z8001/fputw.c -rw-r--r-- 0 steve staff 340 9 Nov 1984 romana/relic/b/lib/libc/stdio/z8001/getw.c -rw-r--r-- 0 steve staff 231 9 Nov 1984 romana/relic/b/lib/libc/stdio/z8001/putw.c -rw-r--r-- 0 steve staff 4906 6 Feb 1991 romana/relic/b/lib/libc/gen/mch/z8001.tar.Z I looked inside the first one, and it gives this set of files, being the library for the syscalls: $ ls z8001/ _brk.s chroot.s getegid.s link.s pause.s signal.s umount.s access.s close.s geteuid.s lock.s pipe.s sload.s unique.s acct.s creat.s getgid.s lseek.s profil.s stat.s unlink.s alarm.s dup.s getpid.s mknod.s ptrace.s stime.s utime.s bpt.s execve.s getuid.s mount.s read.s suload.s wait.s chdir.s fork.s halt.s mpx.s sbrk.c sync.s write.s chmod.s fstat.s ioctl.s nice.s setgid.s times.s chown.s ftime.s kill.s open.s setuid.s umask.s I've not explored any further, but if the kernel source isn't there, I rather suspect the 286 kernel source could be beaten in to some sort of shape to work on the z8000. There some RCS files in there if you wish to explore further. That still leaves you with having to come up with drivers, but there do seem to be (some) technical docs available for the machine. DF From ca6c at firemail.cc Tue Mar 17 23:03:41 2020 From: ca6c at firemail.cc (ca6c at firemail.cc) Date: Tue, 17 Mar 2020 09:03:41 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> Message-ID: <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Doug McIlroy wrote: > dc > > The math library for Bob Morris's variable-precision desk calculator > used backward error analysis to determine the precision necessary at > each step to attain the user-specified precision of the result. In > my software-components talk at the 1968 NATO conference on software > engineering, I posited measurement-standard routines, which could > deliver > results of any desired precision, but did not know how to design one. > dc > still has the only such routines I know of. dc, along with ed and I guess awk if we can put it here, is one of my favorite Unix programs that I use daily. I don't even have a "normal" calculator installed. It just smells like Unix. There is something sexy about reverse Polish notation. I really do encourage everyone reading this to try dc as their "desk calculator" for some time. From akosela at andykosela.com Tue Mar 17 23:30:25 2020 From: akosela at andykosela.com (Andy Kosela) Date: Tue, 17 Mar 2020 14:30:25 +0100 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Message-ID: On 3/17/20, ca6c at firemail.cc wrote: > Doug McIlroy wrote: >> dc >> >> The math library for Bob Morris's variable-precision desk calculator >> used backward error analysis to determine the precision necessary at >> each step to attain the user-specified precision of the result. In >> my software-components talk at the 1968 NATO conference on software >> engineering, I posited measurement-standard routines, which could >> deliver >> results of any desired precision, but did not know how to design one. >> dc >> still has the only such routines I know of. > > dc, along with ed and I guess awk if we can put it here, is one of my > favorite Unix programs that I use daily. I don't even have a "normal" > calculator installed. It just smells like Unix. > > There is something sexy about reverse Polish notation. I really do > encourage everyone reading this to try dc as their "desk calculator" > for some time. So you probably also love Forth, a very underrated language... --Andy From ca6c at firemail.cc Wed Mar 18 00:53:30 2020 From: ca6c at firemail.cc (=?UTF-8?Q?C=C3=A1g?=) Date: Tue, 17 Mar 2020 10:53:30 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Message-ID: Andy Kosela wrote: >> dc, along with ed and I guess awk if we can put it here, is one of my >> favorite Unix programs that I use daily. I don't even have a "normal" >> calculator installed. It just smells like Unix. >> There is something sexy about reverse Polish notation. I really do >> encourage everyone reading this to try dc as their "desk calculator" >> for some time. >> > So you probably also love Forth, a very underrated language... Now I feel bad because I've never heard of it... -- caóc From lm at mcvoy.com Wed Mar 18 00:57:23 2020 From: lm at mcvoy.com (Larry McVoy) Date: Tue, 17 Mar 2020 07:57:23 -0700 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Message-ID: <20200317145723.GF26660@mcvoy.com> On Tue, Mar 17, 2020 at 10:53:30AM -0400, C??g wrote: > Andy Kosela wrote: > > >>dc, along with ed and I guess awk if we can put it here, is one of my > >>favorite Unix programs that I use daily. I don't even have a "normal" > >>calculator installed. It just smells like Unix. > >>There is something sexy about reverse Polish notation. I really do > >>encourage everyone reading this to try dc as their "desk calculator" > >>for some time. > >> > >So you probably also love Forth, a very underrated language... > > Now I feel bad because I've never heard of it... I've written a pile of Forth code and while some people love it, I never warmed up to it. You've seen it if you have used the Sun boot proms, that was all Forth. As much as I don't care for Forth, man do I wish it had become the standard for boot proms, it might not be my cup of tea but I could make it do what I needed it to do. Can't say the same for UEFI, I disable that crap. From usotsuki at buric.co Wed Mar 18 01:40:52 2020 From: usotsuki at buric.co (Steve Nickolas) Date: Tue, 17 Mar 2020 11:40:52 -0400 (EDT) Subject: [TUHS] The most surprising Unix programs In-Reply-To: <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Message-ID: On Tue, 17 Mar 2020, ca6c at firemail.cc wrote: > Doug McIlroy wrote: >> dc >> >> The math library for Bob Morris's variable-precision desk calculator >> used backward error analysis to determine the precision necessary at >> each step to attain the user-specified precision of the result. In >> my software-components talk at the 1968 NATO conference on software >> engineering, I posited measurement-standard routines, which could deliver >> results of any desired precision, but did not know how to design one. dc >> still has the only such routines I know of. > > dc, along with ed and I guess awk if we can put it here, is one of my > favorite Unix programs that I use daily. I don't even have a "normal" > calculator installed. It just smells like Unix. > > There is something sexy about reverse Polish notation. I really do > encourage everyone reading this to try dc as their "desk calculator" > for some time. > I personally prefer bc. Actually, I use GNU's bc on Windows and MS-DOS too. (Unfortunately the traditional version wouldn't work on MS-DOS where it's just a filter to dc.) -uso. From arrigo at alchemistowl.org Wed Mar 18 00:59:59 2020 From: arrigo at alchemistowl.org (Arrigo Triulzi) Date: Tue, 17 Mar 2020 15:59:59 +0100 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <20200317145723.GF26660@mcvoy.com> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> <20200317145723.GF26660@mcvoy.com> Message-ID: <3ABF72C4-3EED-4215-9029-2B291FE18339@alchemistowl.org> On 17 Mar 2020, at 15:57, Larry McVoy wrote: > As much as I don't care for Forth, man do I wish it had become the standard > for boot proms, it might not be my cup of tea but I could make it do what > I needed it to do. Can't say the same for UEFI, I disable that crap. Oh yes, if only… instead we got UEFI. Arrigo From dave at horsfall.org Wed Mar 18 08:28:39 2020 From: dave at horsfall.org (Dave Horsfall) Date: Wed, 18 Mar 2020 09:28:39 +1100 (EST) Subject: [TUHS] The most surprising Unix programs In-Reply-To: <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Message-ID: On Tue, 17 Mar 2020, ca6c at firemail.cc wrote: > There is something sexy about reverse Polish notation. I really do > encourage everyone reading this to try dc as their "desk calculator" for > some time. I've been using my trusty HP-42S for so long that I can hardly remember how to use a "normal" calculator :-) Oh, and I was surprised to learn that "bc" is just a front-end to "dc" when I read the manpage more closely; it is indeed the "Unix Way"[tm]. -- Dave From jon at fourwinds.com Wed Mar 18 10:17:40 2020 From: jon at fourwinds.com (Jon Steinhart) Date: Tue, 17 Mar 2020 17:17:40 -0700 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Message-ID: <202003180017.02I0HeXt327103@darkstar.fourwinds.com> Dave Horsfall writes: > On Tue, 17 Mar 2020, ca6c at firemail.cc wrote: > > > There is something sexy about reverse Polish notation. I really do > > encourage everyone reading this to try dc as their "desk calculator" for > > some time. > > I've been using my trusty HP-42S for so long that I can hardly remember > how to use a "normal" calculator :-) > > Oh, and I was surprised to learn that "bc" is just a front-end to "dc" > when I read the manpage more closely; it is indeed the "Unix Way"[tm]. > > -- Dave I use free42 which is a great 42S emulator that doesn't run out of batteries. From dave at horsfall.org Wed Mar 18 13:28:41 2020 From: dave at horsfall.org (Dave Horsfall) Date: Wed, 18 Mar 2020 14:28:41 +1100 (EST) Subject: [TUHS] The most surprising Unix programs In-Reply-To: <202003180017.02I0HeXt327103@darkstar.fourwinds.com> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> <202003180017.02I0HeXt327103@darkstar.fourwinds.com> Message-ID: On Tue, 17 Mar 2020, Jon Steinhart wrote: [ Probably orta go to COFF ] > I use free42 which is a great 42S emulator that doesn't run out of > batteries. Thanks! Now on my MacBook... -- Dave From steve at quintile.net Wed Mar 18 17:33:58 2020 From: steve at quintile.net (Steve Simon) Date: Wed, 18 Mar 2020 07:33:58 +0000 Subject: [TUHS] calculators Message-ID: i never got on with rpn, even though i am the kind if person who should - i have a little bit of dyslexia, perhaps thats why. when i moved to plan9 i found it included hoc from K&P, just used that for the odd bits of maths i need. -Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Wed Mar 18 18:40:46 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Wed, 18 Mar 2020 02:40:46 -0600 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Message-ID: <202003180840.02I8ekkF013331@freefriends.org> Dave Horsfall wrote: > On Tue, 17 Mar 2020, ca6c at firemail.cc wrote: > > > There is something sexy about reverse Polish notation. I really do > > encourage everyone reading this to try dc as their "desk calculator" for > > some time. > > I've been using my trusty HP-42S for so long that I can hardly remember > how to use a "normal" calculator :-) This might interest you: https://github.com/arnoldrobbins/hp Originally written in Ratfor for the Georgia Tech Software Tools Subsystem and ported to C by yours truly in the mid-80s. I've kept it up to date since then and use it almost daily. Enjoy, Arnold From mike.ab3ap at gmail.com Thu Mar 19 22:26:25 2020 From: mike.ab3ap at gmail.com (Mike Markowski) Date: Thu, 19 Mar 2020 08:26:25 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Message-ID: Warren, can I please be added to COFF, where I know this reply should go? Thanks! On 3/17/20 6:28 PM, Dave Horsfall wrote: > I've been using my trusty HP-42S for so long that I can hardly remember > how to use a "normal" calculator :-) When my classmate's calculator died during an engineering exam, he asked if he could borrow my spare. I handed him my HP 32s and after a minute he whispered, "Where's the equals key?" He gave my calculator back. :-) Mike Markowski From rtomek at ceti.pl Fri Mar 20 06:01:20 2020 From: rtomek at ceti.pl (Tomasz Rola) Date: Thu, 19 Mar 2020 21:01:20 +0100 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Message-ID: <20200319200120.GB16996@tau1.ceti.pl> On Tue, Mar 17, 2020 at 09:03:41AM -0400, ca6c at firemail.cc wrote: > Doug McIlroy wrote: > >dc > > > >The math library for Bob Morris's variable-precision desk > >calculator used backward error analysis to determine the precision > >necessary at each step to attain the user-specified precision of > >the result. In my software-components talk at the 1968 NATO > >conference on software engineering, I posited measurement-standard > >routines, which could deliver results of any desired precision, but > >did not know how to design one. dc still has the only such routines > >I know of. > > dc, along with ed and I guess awk if we can put it here, is one of my > favorite Unix programs that I use daily. I don't even have a "normal" > calculator installed. It just smells like Unix. +1 for (g)awk. After learning it and for many many years treating as "oh just a lines of text processing" tool, I have rediscovered it about a year or two ago. It is (IMHO) a very good alternative language for writing scripts whose complicated control flow might make sh script error-prone and/or ugly. One such script helps me with sound processing - doing some tests for recorded file, composing params for invocation of ecasound (say, check if file needs to be recoded from 48kHz to 44.1kHz etc), and, whoa, invoking ecasound and streaming bytes to stdout for ogg encoder. The script is part of a pipeline, takes multiple filenames as args and cats them one by one, each with the right incantation to ecasound - amplification level, freq changes, all auto, so much modern, look ma no hands :-). Another one is changing color temperature of my monitor. I have found function computing RGB values in someone's code, and the colors were not what I wanted. So the script also has some named profiles in a hash table, where values are a line of three floats, which then gets split()-ed and so on. Overally, I would consider using (g)awk whenever there is a need to invoke some commands based on quirky and complicated criteria. I am sure the scripts could be written in "pure" sh, but something had me step back from this. One task I would be afraid to use awk for, is html processing. Most of html sources I look at nowadays seems discouraging. Extracting anything of value from the mess requires something more potent, I think. > There is something sexy about reverse Polish notation. I really do > encourage everyone reading this to try dc as their "desk calculator" > for some time. For "desk calculator" I am using clisp, a (relatively) lightweight Common Lisp interpreter. I got hooked up after discovering that CL had quite a few numeric types - real rational numbers. making 1/3 and 1/10 easy to have in computer, few floating types, "unlimited" ints, complex numbers. One has to be wary about unintended type conversion, but otherwise I found it very useful for my needs. And for "smart calculator" in a pocket, I would like to use gforth implementation for Android. Once I make myself comfy enough with Forth to not have to poke around manual all the time. -- Regards, Tomasz Rola -- ** A C programmer asked whether computer had Buddha's nature. ** ** As the answer, master did "rm -rif" on the programmer's home ** ** directory. And then the C programmer became enlightened... ** ** ** ** Tomasz Rola mailto:tomasz_rola at bigfoot.com ** From beebe at math.utah.edu Fri Mar 20 06:57:59 2020 From: beebe at math.utah.edu (Nelson H. F. Beebe) Date: Thu, 19 Mar 2020 14:57:59 -0600 Subject: [TUHS] The most surprising Unix programs Message-ID: Tomasz Rola writes on Thu, 19 Mar 2020 21:01:20 +0100 about awk: >> One task I would be afraid to use awk for, is html processing. Most of >> html sources I look at nowadays seems discouraging. Extracting >> anything of value from the mess requires something more potent, I >> think. If you want to tackle raw HTML from abitrary source, then I agree with you: most HTML on the Web is not grammar conformant, there are numerous vendor extensions, and the HTML is hideously idiosynchratic and irregularly formatted. The solution that I adopted 25 years ago was to write a grammar recognizing, but violation lenient, prettyprinter for HTML. It has served well and I use it many times daily for my work in the BibNet Project and TeX User Group bibliography archives, now approaching 1.55 million entries. The latest public release is available here: http://www.math.utah.edu/pub/sgml/ I notice that the last version there is 1.01; I'll get that updated in a couple of days to the latest 1.03 [subject to delays due to major work dislocations due to the virus]. The code should install anywhere in the Unix family without problems: I build and validate it on more than 300 O/Ses in our test farm. With standardized HTML, applying awk is easy, and I have more than 450 awk programs, and 380,000 lines of code, that process publisher metadata to produce rough BibTeX entries that numerous other tools, and some manual editing, turn into clean data for free access on the Web. For some journals, I run a single command of fewer than 15 characters to download Web pages for journal issues for which I do not yet have data, and then a single journal-specific command with no arguments that runs a large shell script with a long pipeline that outputs relatively clean BibTeX that then normally takes me only a couple of minutes to visually validate in an editor session. The major work there is bracing of proper nouns in titles that my software did not already handle, thereby preventing downcasing of those words in the many bibliography styles that do so. I'm on journal announcement lists for many publishers, so I often have new data released to the Web just 5 to 10 minutes after receiving e-mail about new issues. The above-mentioned archives are at http://www.math.utah.edu/pub/bibnet http://www.math.utah.edu/pub/tex/bib http://www.math.utah.edu/pub/tex/bib/index-table.html http://www.math.utah.edu/pub/tex/bib/idx http://www.math.utah.edu/pub/tex/bib/toc They are mirrored at Universität Karlsruhe, Oak Ridge National Laboratory, Sandia National Laboratory, and elsewhere. Like Al Aho, Doug McIlroy, and Arnold Robbins, I'm a huge fan of awk; I believe that I was the first to port it to PDP-10 TOPS-20 and VAX VMS in the mid-1980s, and it is one of the first mandatory tools that I install on any new computer. ------------------------------------------------------------------------------- - Nelson H. F. Beebe Tel: +1 801 581 5254 - - University of Utah FAX: +1 801 581 4148 - - Department of Mathematics, 110 LCB Internet e-mail: beebe at math.utah.edu - - 155 S 1400 E RM 233 beebe at acm.org beebe at computer.org - - Salt Lake City, UT 84112-0090, USA URL: http://www.math.utah.edu/~beebe/ - ------------------------------------------------------------------------------- From rtomek at ceti.pl Fri Mar 20 07:18:33 2020 From: rtomek at ceti.pl (Tomasz Rola) Date: Thu, 19 Mar 2020 22:18:33 +0100 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: Message-ID: <20200319211833.GD16996@tau1.ceti.pl> On Thu, Mar 19, 2020 at 02:57:59PM -0600, Nelson H. F. Beebe wrote: [...] > > If you want to tackle raw HTML from abitrary source, then I agree with > you: most HTML on the Web is not grammar conformant, there are > numerous vendor extensions, and the HTML is hideously idiosynchratic > and irregularly formatted. > > The solution that I adopted 25 years ago was to write a grammar > recognizing, but violation lenient, prettyprinter for HTML. It has > served well and I use it many times daily for my work in the BibNet > Project and TeX User Group bibliography archives, now approaching 1.55 > million entries. The latest public release is available here: > > http://www.math.utah.edu/pub/sgml/ Thank you, I will have a longer look at those archives. My plan so far was to explore html files with CL and Slime (interactive mode for CL inside Emacs), which would allow me to actually find out what I want to be looking for - well, hopefully :-). -- Regards, Tomasz Rola -- ** A C programmer asked whether computer had Buddha's nature. ** ** As the answer, master did "rm -rif" on the programmer's home ** ** directory. And then the C programmer became enlightened... ** ** ** ** Tomasz Rola mailto:tomasz_rola at bigfoot.com ** From dave at horsfall.org Fri Mar 20 07:31:57 2020 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 20 Mar 2020 08:31:57 +1100 (EST) Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Message-ID: On Thu, 19 Mar 2020, Mike Markowski wrote: >> I've been using my trusty HP-42S for so long that I can hardly remember >> how to use a "normal" calculator :-) > > When my classmate's calculator died during an engineering exam, he asked > if he could borrow my spare. I handed him my HP 32s and after a minute > he whispered, "Where's the equals key?" He gave my calculator back. > :-) I did that to a financial controller in a previous life; she was not amused... Hey, it was the only calculator that I had! I could see her helplessly looking for the "=" key, then I took pity on her. -- Dave From arnold at skeeve.com Fri Mar 20 17:14:59 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Fri, 20 Mar 2020 01:14:59 -0600 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: Message-ID: <202003200714.02K7ExMv000801@freefriends.org> "Nelson H. F. Beebe" wrote: > Like Al Aho, Doug McIlroy, and Arnold Robbins, I'm a huge fan of awk; Nice of you to stick me in the same sentence with Al and Doug, but I'm not quite in that league... :-) Arnold From thomas.paulsen at firemail.de Fri Mar 20 17:49:32 2020 From: thomas.paulsen at firemail.de (Thomas Paulsen) Date: Fri, 20 Mar 2020 08:49:32 +0100 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <202003200714.02K7ExMv000801@freefriends.org> References: <202003200714.02K7ExMv000801@freefriends.org> Message-ID: <0c771f1b0064446d0da87a533a586f9e@firemail.de> >Nice of you to stick me in the same sentence with Al and Doug, but >I'm not quite in that league... :-) with that modesty you are in that league.. From arnold at skeeve.com Fri Mar 20 18:18:04 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Fri, 20 Mar 2020 02:18:04 -0600 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <0c771f1b0064446d0da87a533a586f9e@firemail.de> References: <202003200714.02K7ExMv000801@freefriends.org> <0c771f1b0064446d0da87a533a586f9e@firemail.de> Message-ID: <202003200818.02K8I4Ac003630@freefriends.org> "Thomas Paulsen" wrote: > >Nice of you to stick me in the same sentence with Al and Doug, but > >I'm not quite in that league... :-) > with that modesty you are in that league.. > > Very off topic, but I know what Al and Doug have done, and I know what I've done, and I'm not in that league. But thank you. 'nuff said. Arnold From paul at guertin.net Fri Mar 20 21:48:12 2020 From: paul at guertin.net (paul at guertin.net) Date: Fri, 20 Mar 2020 07:48:12 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> Message-ID: <6D9CA6C2-BDF2-4BCA-9503-0F8415C594C9@guertin.net> I teach math in college, and I use an RPN calculator as well (it's just easier). Sometimes, during an exam, a student who forgot to bring their calculator will ask if they can borrow mine. I always say "sure, but you'll regret it" and hand them the calculator. After wasting one or two minutes, they give it back. (Note that I always make sure no calculator is needed for my exams, but it's department policy to authorise non programmable calculators, and it seems to reassure students to have the calculator on the desk, so I don't mind.) On March 19, 2020 5:31:57 PM EDT, Dave Horsfall wrote: >I did that to a financial controller in a previous life; she was not >amused... Hey, it was the only calculator that I had! From jnc at mercury.lcs.mit.edu Sat Mar 21 00:03:08 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Fri, 20 Mar 2020 10:03:08 -0400 (EDT) Subject: [TUHS] The most surprising Unix programs Message-ID: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> > From: Paul Guertin > I teach math in college ... Sometimes, during an exam, a student who > forgot to bring their calculator will ask if they can borrow mine I > always say "sure, but you'll regret it" and hand them the calculator > After wasting one or two minutes, they give it back Maybe I'm being clueless/over-asking, but to me it's appalling that any college student (at least all who have _any_ math requirement at all; not sure how many that is) doesn't know how an RPN calculator works. It's not exactly rocket science, and any reasonably intelligent high-schooler should get it extremely quickly; just tell them it's just a representational thing, number number operator instead of number operator number. I know it's not a key intellectual skill, but it does seem to me to be part of comon intellectual heritage that everyone should know, like musical scales or poetry rhyming. Have you ever considered taking two minutes (literally!) to cover it briefly, just 'someone tried to borrow my RPN calculator, here's the basic idea of how they work'? Noel From rich.salz at gmail.com Sat Mar 21 00:08:50 2020 From: rich.salz at gmail.com (Richard Salz) Date: Fri, 20 Mar 2020 10:08:50 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> Message-ID: A cool thing about tools is that you never know how it will be used. Bach's prelude in C major from Well-Tempered Clavier book I, written in pure (GNU) sed: https://github.com/laserbat/bach.sed -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Sat Mar 21 00:52:57 2020 From: lm at mcvoy.com (Larry McVoy) Date: Fri, 20 Mar 2020 07:52:57 -0700 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> Message-ID: <20200320145257.GD10251@mcvoy.com> On Fri, Mar 20, 2020 at 10:08:50AM -0400, Richard Salz wrote: > A cool thing about tools is that you never know how it will be used. > > Bach's prelude in C major from Well-Tempered Clavier book I, written in > pure (GNU) sed: https://github.com/laserbat/bach.sed If someone understands how this all works, my less capable brain would love an explanation. My dad and my kid have told me that classical music is math but I've never seen it. Seems like this is probably the example I need. From gregg.drwho8 at gmail.com Sat Mar 21 01:03:02 2020 From: gregg.drwho8 at gmail.com (Gregg Levine) Date: Fri, 20 Mar 2020 11:03:02 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <20200320145257.GD10251@mcvoy.com> References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> <20200320145257.GD10251@mcvoy.com> Message-ID: Hello! Actually it is. Some years ago a chap wrote a book comparing Bach, (who has a birth anniversary tomorrow), and that artist who draws illusions and another mathematician as well.It caused an even bigger upset then the OS we discuss here. ----- Gregg C Levine gregg.drwho8 at gmail.com "This signature fought the Time Wars, time and again." On Fri, Mar 20, 2020 at 10:53 AM Larry McVoy wrote: > > On Fri, Mar 20, 2020 at 10:08:50AM -0400, Richard Salz wrote: > > A cool thing about tools is that you never know how it will be used. > > > > Bach's prelude in C major from Well-Tempered Clavier book I, written in > > pure (GNU) sed: https://github.com/laserbat/bach.sed > > If someone understands how this all works, my less capable brain would > love an explanation. My dad and my kid have told me that classical > music is math but I've never seen it. Seems like this is probably > the example I need. From dam at opencsw.org Sat Mar 21 00:58:45 2020 From: dam at opencsw.org (Dagobert Michelsen) Date: Fri, 20 Mar 2020 15:58:45 +0100 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <20200320145257.GD10251@mcvoy.com> References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> <20200320145257.GD10251@mcvoy.com> Message-ID: Hi Larry, Am 20.03.2020 um 15:52 schrieb Larry McVoy : > If someone understands how this all works, my less capable brain would > love an explanation. My dad and my kid have told me that classical > music is math but I've never seen it. Seems like this is probably > the example I need. For this topic I’d recommend the excellent book "Gödel, Escher, Bach: An Eternal Golden Braid“ from Douglas R. Hofstadter which discusses the relationship especially between Bach and math and also gives a nice introduction into logic and philosopy. Best regards — Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From rich.salz at gmail.com Sat Mar 21 01:05:32 2020 From: rich.salz at gmail.com (Richard Salz) Date: Fri, 20 Mar 2020 11:05:32 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> <20200320145257.GD10251@mcvoy.com> Message-ID: It reads a text stream and outputs commands to drive a MIDI player. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chet.ramey at case.edu Sat Mar 21 01:05:45 2020 From: chet.ramey at case.edu (Chet Ramey) Date: Fri, 20 Mar 2020 11:05:45 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> <20200320145257.GD10251@mcvoy.com> Message-ID: On 3/20/20 11:03 AM, Gregg Levine wrote: > Hello! > Actually it is. Some years ago a chap wrote a book comparing Bach, > (who has a birth anniversary tomorrow), and that artist who draws > illusions and another mathematician as well.It caused an even bigger > upset then the OS we discuss here. One of the best books I read in high school. I loved it. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ From cym224 at gmail.com Sat Mar 21 01:07:00 2020 From: cym224 at gmail.com (Nemo) Date: Fri, 20 Mar 2020 11:07:00 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> Message-ID: On 20/03/2020, Noel Chiappa wrote: > > From: Paul Guertin > > > I teach math in college ... Sometimes, during an exam, a student who > > forgot to bring their calculator will ask if they can borrow mine I > > always say "sure, but you'll regret it" and hand them the calculator > > After wasting one or two minutes, they give it back > > [,,,] > [...] and any reasonably intelligent high-schooler should get it > extremely quickly; just tell them it's just a representational thing, > number number operator instead of number operator number. A former colleague -- electronics engineer, to boot -- told me that he refuses to use RPN calculators because one will not always be available and he does not want to become "dependent" on it. Offsetting that, my wife -- who does not work in a technical field -- swears by her 32S (and still has her 35 somewhere). I have my caculators on my Macs all set to RPN. Perhaps a bit mean of me, but when people were befuddled, I used to say that my calculators used "natural" input. N. From gtaylor at tnetconsulting.net Sat Mar 21 01:40:50 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Fri, 20 Mar 2020 09:40:50 -0600 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <6D9CA6C2-BDF2-4BCA-9503-0F8415C594C9@guertin.net> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> <6D9CA6C2-BDF2-4BCA-9503-0F8415C594C9@guertin.net> Message-ID: <211b9d54-573c-05d3-2c60-e15a9fc0b86b@tnetconsulting.net> Aside: I'm sending this reply to TUHS where the message that I'm replying to came from. But i suspect that it should migrate to COFF, which I'm CCing. On 3/20/20 5:48 AM, paul at guertin.net wrote: > I teach math in college, and I use an RPN calculator as well (it's > just easier). Would you please elaborate on "it's just easier"? I'm asking from a point of genuine curiosity. I've heard many say that RPN is easier, or that it takes fewer keys, or otherwise superior to infix notation. But many of the conversations end up somewhat devolving into religious like comments about preferences, despite starting with honest open-minded intentions. (I hope this one doesn't similarly devolve.) I've heard that there are fewer keys to press for RPN, but the example equations presented have been effectively he same. I've heard that RPN is mentally easier. But I apparently don't know enough RPN to be able to think in RPN natively to evaluate myself. I dabble with RPN, including keeping my main calculator app on my smart phone in RPN mode. So I am genuinely interested in understanding why you say that RPN is just easier. > Sometimes, during an exam, a student who forgot to bring their > calculator will ask if they can borrow mine. I always say "sure, but > you'll regret it" and hand them the calculator. After wasting one or > two minutes, they give it back. ~chuckle~ > (Note that I always make sure no calculator is needed for my exams, > but it's department policy to authorise non programmable calculators, > and it seems to reassure students to have the calculator on the desk, > so I don't mind.) > ACK -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From gtaylor at tnetconsulting.net Sat Mar 21 02:07:39 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Fri, 20 Mar 2020 10:07:39 -0600 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> Message-ID: +COFF On 3/20/20 8:03 AM, Noel Chiappa wrote: > Maybe I'm being clueless/over-asking, but to me it's appalling that > any college student (at least all who have _any_ math requirement at > all; not sure how many that is) doesn't know how an RPN calculator > works. I'm sure that there are some people, maybe not the corpus you mention, that have zero clue how an RPN calculator works. But I would expect anybody with a little gumption to be able to poke a few buttons and probably figure out the basic operation, or, ask if they are genuinely confused. > It's not exactly rocket science, and any reasonably intelligent > high-schooler should get it extremely quickly; just tell them it's > just a representational thing, number number operator instead of > number operator number. I agree that RPN is not rocket science. And for basic single operation equations, I think that it's largely interchangeable with infix notation. However, my experience is, as the number of operations goes up, RPN can become more difficult to use. This is likely a mental shortcoming on my part. But it is something that does take tractable mental effort for me to do. For example, let's start with Pythagorean Theorem a² + b² = c² This is relatively easy to enter in infix notation on a typical scientific calculator. However, I have to stop and think about how to enter this on an RPN calculator. I'll take a swing at this, but I might get it wrong, and I don't have anything handy to test at the moment. [a] [enter] [a] [enter] [multiply] [b] [enter] [b] [enter] [multiply] [add] [square root] # to solve for c (12 keys) Conversely infix notation for comparison. [a] [square] [plus] [b] [square] [square root] (6 keys) As I type this, I realize that I'm using higher order operations (square) in infix than I am in RPN. But that probably speaks to my ignorance of RPN. I also realize that this equation does a poor job at demonstrating what I'm trying to convey. — Or perhaps what I'm trying to convey is incorrect. — I had to arrange sub-different parts of the equation so that their results ended up together on the stack for them to be the targets of the operation. I believe this (re)arrangement of the equation is where most of my mental load / objection comes from with RPN. I feel like I have to process the equation before I can tell the calculator to compute the result for me. I don't feel like I have this burden with infix notation. Aside: I firmly believe that computers are supposed to do our bidding, not the other way around. s/computers/calculators/ > I know it's not a key intellectual skill, but it does seem to me to > be part of comon intellectual heritage that everyone should know, > like musical scales or poetry rhyming. Have you ever considered > taking two minutes (literally!) to cover it briefly, just 'someone > tried to borrow my RPN calculator, here's the basic idea of how they > work'? I'm confident that 80% of people, more of the corpus you describe, could use an RPN calculator to do simple equations. But I would not be surprised if many found that the re-arrangement of equations to being RPN friendly would simply forego the RPN calculator for simpler arithmetic operations. I think some of it is a mental question: Which has more mental load, doing the annoying arithmetic or re-arranging to use RPN. I believe that for the simpler of the arithmetic operations, RPN is going to be more difficult. All of this being said, I'd love to have someone lay out points and / or counterpoints to my understanding. -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From jon at fourwinds.com Sat Mar 21 02:40:53 2020 From: jon at fourwinds.com (Jon Steinhart) Date: Fri, 20 Mar 2020 09:40:53 -0700 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <211b9d54-573c-05d3-2c60-e15a9fc0b86b@tnetconsulting.net> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> <6D9CA6C2-BDF2-4BCA-9503-0F8415C594C9@guertin.net> <211b9d54-573c-05d3-2c60-e15a9fc0b86b@tnetconsulting.net> Message-ID: <202003201640.02KGerlG470796@darkstar.fourwinds.com> Grant Taylor via TUHS writes: > > I'm asking from a point of genuine curiosity. I've heard many say that > RPN is easier, or that it takes fewer keys, or otherwise superior to > infix notation. But many of the conversations end up somewhat devolving > > into religious like comments about preferences, despite starting with > honest open-minded intentions. (I hope this one doesn't similarly devolve.) I've been a RPN fan since I got my first HP-45 when I was a sophomore in college. I see the difference between RPN and algebraic notation as follows. RPN is much easier when thinking on the fly; algabraic is much easier when copying equations off of a page. I think that this mainly comes from not having to worry about parentheses with RPN. Moving things around on the stack when having to rearrange is easier than having to move things in and out of registers. Jon From gtaylor at tnetconsulting.net Sat Mar 21 03:23:52 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Fri, 20 Mar 2020 11:23:52 -0600 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <202003201640.02KGerlG470796@darkstar.fourwinds.com> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> <6D9CA6C2-BDF2-4BCA-9503-0F8415C594C9@guertin.net> <211b9d54-573c-05d3-2c60-e15a9fc0b86b@tnetconsulting.net> <202003201640.02KGerlG470796@darkstar.fourwinds.com> Message-ID: <0b0d0ba3-7eae-a844-cc9a-ae542edb302b@tnetconsulting.net> On 3/20/20 10:40 AM, Jon Steinhart wrote: > RPN is much easier when thinking on the fly; algabraic is much easier > when copying equations off of a page. Would you humor me with an example of what you mean by "thinking on the fly"? Either I'm not understanding you or we think differently. I think I understand the algabraic copying. > I think that this mainly comes from not having to worry about > parentheses with RPN. Moving things around on the stack when having > to rearrange is easier than having to move things in and out of > registers. I'm not following. With my limited understanding of RPN, I would not currently dare alter the stack. Perhaps that's part of why I find RPN to be more of a computational burden in needing to rearrange the equation to align with stack order. -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From rdm at cfcl.com Sat Mar 21 04:43:05 2020 From: rdm at cfcl.com (Rich Morin) Date: Fri, 20 Mar 2020 11:43:05 -0700 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <202003201640.02KGerlG470796@darkstar.fourwinds.com> References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <7ec47fd97b1a3d383ffed428f21f5287@firemail.cc> <6D9CA6C2-BDF2-4BCA-9503-0F8415C594C9@guertin.net> <211b9d54-573c-05d3-2c60-e15a9fc0b86b@tnetconsulting.net> <202003201640.02KGerlG470796@darkstar.fourwinds.com> Message-ID: Back in the 70's, I bought an HP-67 for some ridiculous amount of money. I only wrote one significant program on it, but I think folks here might find the idea interesting. My X-Ray astronomy project at NRL (HEAO A1) was using a CDC 3800 for data analysis. The machine had 48-bit words and used octal to display binary values. I decided to write a program to convert the binary representation of a floating point number into the numeric value. For starters, it had to accept a pair of 24-bit values, expressed in (mock) octal. That is, when I entered 12345, it was actually 12345(8), not 12345(10). So, the program had to turn these values back into real integers. It then pulled out the exponent and mantissa, did the appropriate calculations, and displayed the floating-point value. It worked, but it wasn't all that fast. As I recall, it took about 30 seconds... -r From athornton at gmail.com Sat Mar 21 05:03:52 2020 From: athornton at gmail.com (Adam Thornton) Date: Fri, 20 Mar 2020 12:03:52 -0700 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> Message-ID: <19339755-18E7-488A-8B86-F128A28A36B3@gmail.com> > On Mar 20, 2020, at 8:07 AM, Nemo wrote: > > Offsetting that, my wife -- who does not work in a technical field -- > swears by her 32S (and still has her 35 somewhere). I have my > caculators on my Macs all set to RPN. You can have my HP-28S when you pry it from my cold dead hands. Adam From egbegb2 at gmail.com Sat Mar 21 06:13:57 2020 From: egbegb2 at gmail.com (Ed Bradford) Date: Fri, 20 Mar 2020 15:13:57 -0500 Subject: [TUHS] I made a tar tape Message-ID: When I left BTL in 1983, I made a tar tape. A number of years later I translated the tape into a file. Only recently have I wandered through it. I don't know how many people remember Ron Hardin in the Columbus BTL location. He was one of the smartest guys I ever met. There are lot of Ron Hardin stories. One of his creations (as far as I know he authored it) was a program to create Memorandums For File -- technical memorandums. My tar tape scooped up festoon. To this day it compiles and runs happily on Windows 10. It was written in 1978 or thereabouts. Here is an example output: bin$ festoon.exe .TL No Worthynesses .AU "C. C. Festoon" CCF Headquarters 1584734291 .AS A restriction had been being amicated by a convenience at the inclusion. .AE .MT "MEMORANDUM FOR COAT LOCKER" .hy 1 On this occasion, no team responsibilities could have polyesced a renewed emphasis. A friction had penated an activation. At the present moment in time, an undue number of good progresses being collected together with the populations were being proportionately fideated by the fact that there was a data stream which was transenniesced by an issuance being joined together with these team re sponsibilities, because natural basises have been veriating a partitioning. The supplementary work should be conclusively quinquepolyated by a well defined interfacing. A sophisticatedness by a schedule is operated by a nature in conflict with a correspondence under some serious discussi ons. It is within the realm of possibility that the effectiveness had vicfacesced a schedule, but there was not a necessary background information which is being testesced by a strong interest, and a statistical accuracy was tempoesced by the preparation. It should be noted that a joint partnership very repeatedly aidioated this publication of a centralized organization. Due to the fact that there is a simplification which simply enniesced a process, a new technology is fluxesced from monorogatities. It is of the utmost importance that an insurance could be putated by an assumption. A major advance centered about a deficiency octocessates an important outcome. .P An effectation would extramicroate to the situation. A complete revision gravated a direction. Inasmuch as there was not a potential usefulness that cedeates by the timely delivery, a consideration centered around a technique was monofortated by an integration: .BL .LI There is a not unclear meterdom which had risiesced an occasion. .LE .P A clamstress of this enclosedness is cludescing the hemidormity. .P To arrive at an approximation, a large quantity had been chromated by a strong feeling. Moreover, that idea sharing was lusated by a current proposal. Anytime that the final outcomes had been very firmly unpathesced by not unphilaible reasonable compromises, no serious concerns might be being sacrated by internal establishments for the basic objectives in back of a full utili zation. .P As a consequence of the fact that a total effect might vacate an easily situational beneficial assistance, the apparent provisioning being effectuated by a continuing difference can have protenesced a realization of an underly ing purpose. A different doubtful important outcome is cludated by a capkin. A rationale had fortated attachments. Moreover, this assumption had nilcoresced the continuing study. .P .H 1 "An Easily Added Basic Assumption Being Joined Together With A Concept Stage" There is not an impediment which neoated a restriction, therefore. A couple utilizations could morsate a great similarity at considerable difficulties, but an input is primescing the concept activities, and a growing importance was hemicisesced by that beneficial assistance. In the same connection, these extremenesses are rather usefully ultralucesced by directions. .SG .NS 0 C. R. Glitch S. A. Hobble R. S. Limn M. Shayegan .NE Ed Bradford, Ph.D. Physics, retired from IBM BTL 1976-1983 -- Advice is judged by results, not by intentions. Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From mah at mhorton.net Sat Mar 21 07:26:51 2020 From: mah at mhorton.net (Mary Ann Horton) Date: Fri, 20 Mar 2020 14:26:51 -0700 Subject: [TUHS] I made a tar tape In-Reply-To: References: Message-ID: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> I remember Ron! He was quite a character. And I remember festoon. Was it he, or Bob Flandrena, that gave us the holiday poetical and phonetical treat:     Tweeze Denied Beef Workers Isthmus (Say it out loud fast, with a pause in the middle of "Workers", to get it). I think I have it on paper somewhere, but no soft copy.     Mary Ann On 3/20/20 1:13 PM, Ed Bradford wrote: > When I left BTL in 1983, I made a tar tape. A number of years later I > translated the tape into a file. Only recently have I wandered through it. > > I don't know how many people remember Ron Hardin in the Columbus BTL > location. He was one of the smartest guys I ever met. There are lot of > Ron Hardin stories. One of his creations (as far as I know he authored > it) was a program to create Memorandums For File -- technical > memorandums. My tar tape scooped up festoon. To this day it compiles > and runs happily on Windows 10. It was written in 1978 or thereabouts. > Here is an example output: > > bin$ festoon.exe .TL No Worthynesses .AU "C. C. Festoon" CCF > Headquarters 1584734291 .AS A restriction had been being amicated by a > convenience at the inclusion. .AE .MT "MEMORANDUM FOR COAT LOCKER" .hy > 1 On this occasion, no team responsibilities could have polyesced a > renewed emphasis. A friction had penated an activation. At the present > moment in time, an undue number of good progresses being collected > together with the populations were being proportionately fideated by > the fact that there was a data stream which was transenniesced by an > issuance being joined together with these team re sponsibilities, > because natural basises have been veriating a partitioning. The > supplementary work should be conclusively quinquepolyated by a well > defined interfacing. A sophisticatedness by a schedule is operated by > a nature in conflict with a correspondence under some serious discussi > ons. It is within the realm of possibility that the effectiveness had > vicfacesced a schedule, but there was not a necessary background > information which is being testesced by a strong interest, and a > statistical accuracy was tempoesced by the preparation. It should be > noted that a joint partnership very repeatedly aidioated this > publication of a centralized organization. Due to the fact that there > is a simplification which simply enniesced a process, a new technology > is fluxesced from monorogatities. It is of the utmost importance that > an insurance could be putated by an assumption. A major advance > centered about a deficiency octocessates an important outcome. .P An > effectation would extramicroate to the situation. A complete revision > gravated a direction. Inasmuch as there was not a potential usefulness > that cedeates by the timely delivery, a consideration centered around > a technique was monofortated by an integration: .BL .LI There is a not > unclear meterdom which had risiesced an occasion. .LE .P A clamstress > of this enclosedness is cludescing the hemidormity. .P To arrive at an > approximation, a large quantity had been chromated by a strong > feeling. Moreover, that idea sharing was lusated by a current > proposal. Anytime that the final outcomes had been very firmly > unpathesced by not unphilaible reasonable compromises, no serious > concerns might be being sacrated by internal establishments for the > basic objectives in back of a full utili zation. .P As a consequence > of the fact that a total effect might vacate an easily situational > beneficial assistance, the apparent provisioning being effectuated by > a continuing difference can have protenesced a realization of an > underly ing purpose. A different doubtful important outcome is > cludated by a capkin. A rationale had fortated attachments. Moreover, > this assumption had nilcoresced the continuing study. .P .H 1 "An > Easily Added Basic Assumption Being Joined Together With A Concept > Stage" There is not an impediment which neoated a restriction, > therefore. A couple utilizations could morsate a great similarity at > considerable difficulties, but an input is primescing the concept > activities, and a growing importance was hemicisesced by that > beneficial assistance. In the same connection, these extremenesses are > rather usefully ultralucesced by directions. .SG .NS 0 C. R. Glitch S. > A. Hobble R. S. Limn M. Shayegan > .NE > > Ed Bradford, Ph.D. Physics, retired from IBM > BTL 1976-1983 > > -- > Advice is judged by results, not by intentions. >   Cicero > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Sat Mar 21 07:57:21 2020 From: dave at horsfall.org (Dave Horsfall) Date: Sat, 21 Mar 2020 08:57:21 +1100 (EST) Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> Message-ID: On Fri, 20 Mar 2020, Richard Salz wrote: > A cool thing about tools is that you never know how it will be used. Indeed. I once wrote a shell script that basically acted as an n-way "cut" i.e. you specify the columns you wanted to extract and they ended up on the corresponding file descriptors. Ran into all sorts of bugs, as the "obvious" solution didn't work. > Bach's prelude in C major from Well-Tempered Clavier book I, written in pure > (GNU) sed: https://github.com/laserbat/bach.sed I have the Towers of Hanoi written in sendmail.cf, meaning that Sendmail is Turing-complete :-) I can't remember who wrote it. -- Dave From mike.ab3ap at gmail.com Sat Mar 21 08:04:58 2020 From: mike.ab3ap at gmail.com (Mike Markowski) Date: Fri, 20 Mar 2020 18:04:58 -0400 Subject: [TUHS] I made a tar tape In-Reply-To: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> Message-ID: That took me a few tries - love it! :-) - Mike Markowski On 3/20/20 5:26 PM, Mary Ann Horton wrote: > I remember Ron! He was quite a character. And I remember festoon. > > Was it he, or Bob Flandrena, that gave us the holiday poetical and > phonetical treat: > >     Tweeze Denied Beef Workers Isthmus > > (Say it out loud fast, with a pause in the middle of "Workers", to get it). > > I think I have it on paper somewhere, but no soft copy. > >     Mary Ann From dave at horsfall.org Sat Mar 21 08:06:54 2020 From: dave at horsfall.org (Dave Horsfall) Date: Sat, 21 Mar 2020 09:06:54 +1100 (EST) Subject: [TUHS] The most surprising Unix programs In-Reply-To: <20200320145257.GD10251@mcvoy.com> References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> <20200320145257.GD10251@mcvoy.com> Message-ID: On Fri, 20 Mar 2020, Larry McVoy wrote: >> Bach's prelude in C major from Well-Tempered Clavier book I, written in >> pure (GNU) sed: https://github.com/laserbat/bach.sed > > If someone understands how this all works, my less capable brain would > love an explanation. My dad and my kid have told me that classical > music is math but I've never seen it. Seems like this is probably > the example I need. "Gödel, Escher, Bach: An Eternal Golden Braid" by Douglas Hofstadter would be a good introduction too. -- Dave From mike.ab3ap at gmail.com Sat Mar 21 08:09:46 2020 From: mike.ab3ap at gmail.com (Mike Markowski) Date: Fri, 20 Mar 2020 18:09:46 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> <20200320145257.GD10251@mcvoy.com> Message-ID: <21922bdc-9fa8-1bd9-7770-668bb09854d4@gmail.com> I quoted Hofstadter's sage words in my dissertation intro: Hofstadter’s Law: It always takes longer than you expect, even when you take into account Hofstadter’s Law. Mike Markowski On 3/20/20 10:58 AM, Dagobert Michelsen wrote: > ...For this topic I’d recommend the excellent book "Gödel, Escher, Bach: > An Eternal Golden Braid“ from Douglas R. Hofstadter which discusses > the relationship especially between Bach and math and also gives a > nice introduction into logic and philosopy. > > Best regards — Dago > From robpike at gmail.com Sat Mar 21 08:49:24 2020 From: robpike at gmail.com (Rob Pike) Date: Sat, 21 Mar 2020 09:49:24 +1100 Subject: [TUHS] I made a tar tape In-Reply-To: References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> Message-ID: I believe Festoon was covered in a Sci Am article by Dewdney. It included Mark V Shaney, some computer-generated poetry, and other horrors, including computer-generated Baudrillard. -rob On Sat, Mar 21, 2020 at 9:05 AM Mike Markowski wrote: > > That took me a few tries - love it! :-) - Mike Markowski > > On 3/20/20 5:26 PM, Mary Ann Horton wrote: > > I remember Ron! He was quite a character. And I remember festoon. > > > > Was it he, or Bob Flandrena, that gave us the holiday poetical and > > phonetical treat: > > > > Tweeze Denied Beef Workers Isthmus > > > > (Say it out loud fast, with a pause in the middle of "Workers", to get it). > > > > I think I have it on paper somewhere, but no soft copy. > > > > Mary Ann From doug at cs.dartmouth.edu Sat Mar 21 09:15:00 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Fri, 20 Mar 2020 19:15:00 -0400 Subject: [TUHS] RPN (was The most surprising Unix programs) Message-ID: <202003202315.02KNF088305876@coolidge.cs.dartmouth.edu> Sorry to flog this topic, but the two examples below are an unfair comparison. What happened to the multiplications in the second? And two of the [enter]s in the first are unnecessary. Ironically three of the four operations in the second are actually reverse Polish! If you had to utter sqrt first, as we do on paper and in speech, things wouldn't look so great. [a] [enter] [a] [enter] [multiply] [b] [enter] [b] [enter] [multiply] [add] [square root] [a] [square] [plus] [b] [square] [square root] Doug From dave at horsfall.org Sat Mar 21 09:20:18 2020 From: dave at horsfall.org (Dave Horsfall) Date: Sat, 21 Mar 2020 10:20:18 +1100 (EST) Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> Message-ID: On Mon, 16 Mar 2020, Rob Pike wrote: > There's a copy in my github repo for it, along w2006.txt and the salt > file. > > https://github.com/robpike/typo/tree/master/unix Thanks. Errkk... I see that comments weren't exactly his strong point :-) And someone must've added those #include lines, because those files didn't exist way back then. -- Dave From toby at telegraphics.com.au Sat Mar 21 09:35:30 2020 From: toby at telegraphics.com.au (Toby Thain) Date: Fri, 20 Mar 2020 19:35:30 -0400 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> Message-ID: <59d6753b-3d98-5fb8-162a-3d0781ea4563@telegraphics.com.au> On 2020-03-20 7:20 PM, Dave Horsfall wrote: > On Mon, 16 Mar 2020, Rob Pike wrote: > >> There's a copy in my github repo for it, along w2006.txt and the salt >> file. >> >> https://github.com/robpike/typo/tree/master/unix > > Thanks. > > Errkk...  I see that comments weren't exactly his strong point :-)  And *looks at the definition of rand()* > someone must've added those #include lines, because those files didn't > exist way back then. > > -- Dave From jpl.jpl at gmail.com Sat Mar 21 10:16:00 2020 From: jpl.jpl at gmail.com (John P. Linderman) Date: Fri, 20 Mar 2020 20:16:00 -0400 Subject: [TUHS] I made a tar tape In-Reply-To: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> Message-ID: If you enjoyed that, try this, apparently done in 1940. Ron and Bob were something else. I recall Bob "running for DMTS" by handing out dimes in the Murray Hill cafeteria. On Fri, Mar 20, 2020 at 5:28 PM Mary Ann Horton wrote: > I remember Ron! He was quite a character. And I remember festoon. > > Was it he, or Bob Flandrena, that gave us the holiday poetical and > phonetical treat: > > Tweeze Denied Beef Workers Isthmus > > (Say it out loud fast, with a pause in the middle of "Workers", to get it). > > I think I have it on paper somewhere, but no soft copy. > > Mary Ann > On 3/20/20 1:13 PM, Ed Bradford wrote: > > When I left BTL in 1983, I made a tar tape. A number of years later I > translated the tape into a file. Only recently have I wandered through it. > > I don't know how many people remember Ron Hardin in the Columbus BTL > location. He was one of the smartest guys I ever met. There are lot of Ron > Hardin stories. One of his creations (as far as I know he authored it) was > a program to create Memorandums For File -- technical memorandums. My tar > tape scooped up festoon. To this day it compiles and runs happily on > Windows 10. It was written in 1978 or thereabouts. Here is an example > output: > > bin$ festoon.exe .TL No Worthynesses .AU "C. C. Festoon" CCF Headquarters > 1584734291 .AS A restriction had been being amicated by a convenience at > the inclusion. .AE .MT "MEMORANDUM FOR COAT LOCKER" .hy 1 On this occasion, > no team responsibilities could have polyesced a renewed emphasis. A > friction had penated an activation. At the present moment in time, an undue > number of good progresses being collected together with the populations > were being proportionately fideated by the fact that there was a data > stream which was transenniesced by an issuance being joined together with > these team re sponsibilities, because natural basises have been veriating a > partitioning. The supplementary work should be conclusively quinquepolyated > by a well defined interfacing. A sophisticatedness by a schedule is > operated by a nature in conflict with a correspondence under some serious > discussi ons. It is within the realm of possibility that the effectiveness > had vicfacesced a schedule, but there was not a necessary background > information which is being testesced by a strong interest, and a > statistical accuracy was tempoesced by the preparation. It should be noted > that a joint partnership very repeatedly aidioated this publication of a > centralized organization. Due to the fact that there is a simplification > which simply enniesced a process, a new technology is fluxesced from > monorogatities. It is of the utmost importance that an insurance could be > putated by an assumption. A major advance centered about a deficiency > octocessates an important outcome. .P An effectation would extramicroate to > the situation. A complete revision gravated a direction. Inasmuch as there > was not a potential usefulness that cedeates by the timely delivery, a > consideration centered around a technique was monofortated by an > integration: .BL .LI There is a not unclear meterdom which had risiesced an > occasion. .LE .P A clamstress of this enclosedness is cludescing the > hemidormity. .P To arrive at an approximation, a large quantity had been > chromated by a strong feeling. Moreover, that idea sharing was lusated by a > current proposal. Anytime that the final outcomes had been very firmly > unpathesced by not unphilaible reasonable compromises, no serious concerns > might be being sacrated by internal establishments for the basic objectives > in back of a full utili zation. .P As a consequence of the fact that a > total effect might vacate an easily situational beneficial assistance, the > apparent provisioning being effectuated by a continuing difference can have > protenesced a realization of an underly ing purpose. A different doubtful > important outcome is cludated by a capkin. A rationale had fortated > attachments. Moreover, this assumption had nilcoresced the continuing > study. .P .H 1 "An Easily Added Basic Assumption Being Joined Together With > A Concept Stage" There is not an impediment which neoated a restriction, > therefore. A couple utilizations could morsate a great similarity at > considerable difficulties, but an input is primescing the concept > activities, and a growing importance was hemicisesced by that beneficial > assistance. In the same connection, these extremenesses are rather usefully > ultralucesced by directions. .SG .NS 0 C. R. Glitch S. A. Hobble R. S. Limn > M. Shayegan > .NE > > Ed Bradford, Ph.D. Physics, retired from IBM > BTL 1976-1983 > > -- > Advice is judged by results, not by intentions. > Cicero > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robpike at gmail.com Sat Mar 21 10:28:57 2020 From: robpike at gmail.com (Rob Pike) Date: Sat, 21 Mar 2020 11:28:57 +1100 Subject: [TUHS] I made a tar tape In-Reply-To: References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> Message-ID: Anemometer cur chiffon. See if that rings any bells. -rob On Sat, Mar 21, 2020 at 11:17 AM John P. Linderman wrote: > > If you enjoyed that, try this, apparently done in 1940. > > Ron and Bob were something else. I recall Bob "running for DMTS" by handing out dimes in the Murray Hill cafeteria. > > On Fri, Mar 20, 2020 at 5:28 PM Mary Ann Horton wrote: >> >> I remember Ron! He was quite a character. And I remember festoon. >> >> Was it he, or Bob Flandrena, that gave us the holiday poetical and phonetical treat: >> >> Tweeze Denied Beef Workers Isthmus >> >> (Say it out loud fast, with a pause in the middle of "Workers", to get it). >> >> I think I have it on paper somewhere, but no soft copy. >> >> Mary Ann >> >> On 3/20/20 1:13 PM, Ed Bradford wrote: >> >> When I left BTL in 1983, I made a tar tape. A number of years later I translated the tape into a file. Only recently have I wandered through it. >> >> I don't know how many people remember Ron Hardin in the Columbus BTL location. He was one of the smartest guys I ever met. There are lot of Ron Hardin stories. One of his creations (as far as I know he authored it) was a program to create Memorandums For File -- technical memorandums. My tar tape scooped up festoon. To this day it compiles and runs happily on Windows 10. It was written in 1978 or thereabouts. Here is an example output: >> >> bin$ festoon.exe .TL No Worthynesses .AU "C. C. Festoon" CCF Headquarters 1584734291 .AS A restriction had been being amicated by a convenience at the inclusion. .AE .MT "MEMORANDUM FOR COAT LOCKER" .hy 1 On this occasion, no team responsibilities could have polyesced a renewed emphasis. A friction had penated an activation. At the present moment in time, an undue number of good progresses being collected together with the populations were being proportionately fideated by the fact that there was a data stream which was transenniesced by an issuance being joined together with these team re sponsibilities, because natural basises have been veriating a partitioning. The supplementary work should be conclusively quinquepolyated by a well defined interfacing. A sophisticatedness by a schedule is operated by a nature in conflict with a correspondence under some serious discussi ons. It is within the realm of possibility that the effectiveness had vicfacesced a schedule, but there was not a necessary background information which is being testesced by a strong interest, and a statistical accuracy was tempoesced by the preparation. It should be noted that a joint partnership very repeatedly aidioated this publication of a centralized organization. Due to the fact that there is a simplification which simply enniesced a process, a new technology is fluxesced from monorogatities. It is of the utmost importance that an insurance could be putated by an assumption. A major advance centered about a deficiency octocessates an important outcome. .P An effectation would extramicroate to the situation. A complete revision gravated a direction. Inasmuch as there was not a potential usefulness that cedeates by the timely delivery, a consideration centered around a technique was monofortated by an integration: .BL .LI There is a not unclear meterdom which had risiesced an occasion. .LE .P A clamstress of this enclosedness is cludescing the hemidormity. .P To arrive at an approximation, a large quantity had been chromated by a strong feeling. Moreover, that idea sharing was lusated by a current proposal. Anytime that the final outcomes had been very firmly unpathesced by not unphilaible reasonable compromises, no serious concerns might be being sacrated by internal establishments for the basic objectives in back of a full utili zation. .P As a consequence of the fact that a total effect might vacate an easily situational beneficial assistance, the apparent provisioning being effectuated by a continuing difference can have protenesced a realization of an underly ing purpose. A different doubtful important outcome is cludated by a capkin. A rationale had fortated attachments. Moreover, this assumption had nilcoresced the continuing study. .P .H 1 "An Easily Added Basic Assumption Being Joined Together With A Concept Stage" There is not an impediment which neoated a restriction, therefore. A couple utilizations could morsate a great similarity at considerable difficulties, but an input is primescing the concept activities, and a growing importance was hemicisesced by that beneficial assistance. In the same connection, these extremenesses are rather usefully ultralucesced by directions. .SG .NS 0 C. R. Glitch S. A. Hobble R. S. Limn M. Shayegan >> .NE >> >> Ed Bradford, Ph.D. Physics, retired from IBM >> BTL 1976-1983 >> >> -- >> Advice is judged by results, not by intentions. >> Cicero >> From robpike at gmail.com Sat Mar 21 10:34:27 2020 From: robpike at gmail.com (Rob Pike) Date: Sat, 21 Mar 2020 11:34:27 +1100 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <202003132331.02DNVaxN061501@tahoe.cs.Dartmouth.EDU> <0538034f-98cb-1063-b94d-e7b77b548d6d@gmail.com> Message-ID: Oops, that was me. Leftover from a porting attempt. Will fix. -rob On Sat, Mar 21, 2020 at 10:21 AM Dave Horsfall wrote: > > On Mon, 16 Mar 2020, Rob Pike wrote: > > > There's a copy in my github repo for it, along w2006.txt and the salt > > file. > > > > https://github.com/robpike/typo/tree/master/unix > > Thanks. > > Errkk... I see that comments weren't exactly his strong point :-) And > someone must've added those #include lines, because those files didn't > exist way back then. > > -- Dave From robpike at gmail.com Sat Mar 21 10:40:44 2020 From: robpike at gmail.com (Rob Pike) Date: Sat, 21 Mar 2020 11:40:44 +1100 Subject: [TUHS] RPN (was The most surprising Unix programs) In-Reply-To: <202003202315.02KNF088305876@coolidge.cs.dartmouth.edu> References: <202003202315.02KNF088305876@coolidge.cs.dartmouth.edu> Message-ID: Oh, flog away. RPN works for me because the thing on the screen is usually the value so far. I get to think about what to do with what I've got in my hand. Equation-driven expression doesn't flow like that. As a calculator, as opposed to an equation evaluator, it's RPN for me. -rob From gtaylor at tnetconsulting.net Sat Mar 21 10:43:08 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Fri, 20 Mar 2020 18:43:08 -0600 Subject: [TUHS] RPN (was The most surprising Unix programs) In-Reply-To: <202003202315.02KNF088305876@coolidge.cs.dartmouth.edu> References: <202003202315.02KNF088305876@coolidge.cs.dartmouth.edu> Message-ID: <638f3041-73e5-639f-5e2c-8387978a38d5@spamtrap.tnetconsulting.net> On 3/20/20 5:15 PM, Doug McIlroy wrote: > Sorry to flog this topic, but the two examples below are an unfair > comparison. What happened to the multiplications in the second? Are you referring to the second {a,b} and the subsequent multiply? (I'm presuming so until corrected otherwise.) I think that is addressed by the square operation. > And two of the [enter]s in the first are unnecessary. Which two? The enter before the multiplies? Seeing as how I'm trying to learn from the collective experience of TUHS, would you please elaborate or reproduce what you believe to be the correct key sequences? > Ironically three of the four operations in the second are actually > reverse Polish! If you had to utter sqrt first, as we do on paper > and in speech, things wouldn't look so great. Hum.... :-/ -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From jnc at mercury.lcs.mit.edu Sat Mar 21 11:12:05 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Fri, 20 Mar 2020 21:12:05 -0400 (EDT) Subject: [TUHS] The most surprising Unix programs Message-ID: <20200321011205.3B0D318C073@mercury.lcs.mit.edu> > From: Dagobert Michelsen > the excellent book "G=C3=B6del, Escher, Bach: An Eternal Golden Braid" > from Douglas R. Hofstadter which also gives a nice introduction into > logic and philosopy. IIRC, the focus of the book is how systems made out of simple components can exhibit complex behaviours; in particular, how information-processing systems can come to develop self-awareness. > From: Chet Ramey > One of the best books I read in high school. A book on a very similar topic to GEB, which was _extremely_ important in developing my understanding of how the universe came to be, is "Recursive Universe", by William Poundstone, which I recommend very highly to everyone here. It's still in print, which is really good, because it's not as well known as it should (IMO) be. It uses an analogy with Conway's Life to explain how the large-scale structure of the universe can develop from a random initial state. Buy it now! Noel From ken at google.com Sat Mar 21 11:14:09 2020 From: ken at google.com (Ken Thompson) Date: Fri, 20 Mar 2020 18:14:09 -0700 Subject: [TUHS] I made a tar tape In-Reply-To: References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> Message-ID: two hardin's sprint to mind. 1. phonetix 2. dont put your hands in tight places, remember you only have two. On Fri, Mar 20, 2020 at 5:29 PM Rob Pike wrote: > Anemometer cur chiffon. > > See if that rings any bells. > > -rob > > On Sat, Mar 21, 2020 at 11:17 AM John P. Linderman > wrote: > > > > If you enjoyed that, try this, apparently done in 1940. > > > > Ron and Bob were something else. I recall Bob "running for DMTS" by > handing out dimes in the Murray Hill cafeteria. > > > > On Fri, Mar 20, 2020 at 5:28 PM Mary Ann Horton wrote: > >> > >> I remember Ron! He was quite a character. And I remember festoon. > >> > >> Was it he, or Bob Flandrena, that gave us the holiday poetical and > phonetical treat: > >> > >> Tweeze Denied Beef Workers Isthmus > >> > >> (Say it out loud fast, with a pause in the middle of "Workers", to get > it). > >> > >> I think I have it on paper somewhere, but no soft copy. > >> > >> Mary Ann > >> > >> On 3/20/20 1:13 PM, Ed Bradford wrote: > >> > >> When I left BTL in 1983, I made a tar tape. A number of years later I > translated the tape into a file. Only recently have I wandered through it. > >> > >> I don't know how many people remember Ron Hardin in the Columbus BTL > location. He was one of the smartest guys I ever met. There are lot of Ron > Hardin stories. One of his creations (as far as I know he authored it) was > a program to create Memorandums For File -- technical memorandums. My tar > tape scooped up festoon. To this day it compiles and runs happily on > Windows 10. It was written in 1978 or thereabouts. Here is an example > output: > >> > >> bin$ festoon.exe .TL No Worthynesses .AU "C. C. Festoon" CCF > Headquarters 1584734291 .AS A restriction had been being amicated by a > convenience at the inclusion. .AE .MT "MEMORANDUM FOR COAT LOCKER" .hy 1 On > this occasion, no team responsibilities could have polyesced a renewed > emphasis. A friction had penated an activation. At the present moment in > time, an undue number of good progresses being collected together with the > populations were being proportionately fideated by the fact that there was > a data stream which was transenniesced by an issuance being joined together > with these team re sponsibilities, because natural basises have been > veriating a partitioning. The supplementary work should be conclusively > quinquepolyated by a well defined interfacing. A sophisticatedness by a > schedule is operated by a nature in conflict with a correspondence under > some serious discussi ons. It is within the realm of possibility that the > effectiveness had vicfacesced a schedule, but there was not a necessary > background information which is being testesced by a strong interest, and a > statistical accuracy was tempoesced by the preparation. It should be noted > that a joint partnership very repeatedly aidioated this publication of a > centralized organization. Due to the fact that there is a simplification > which simply enniesced a process, a new technology is fluxesced from > monorogatities. It is of the utmost importance that an insurance could be > putated by an assumption. A major advance centered about a deficiency > octocessates an important outcome. .P An effectation would extramicroate to > the situation. A complete revision gravated a direction. Inasmuch as there > was not a potential usefulness that cedeates by the timely delivery, a > consideration centered around a technique was monofortated by an > integration: .BL .LI There is a not unclear meterdom which had risiesced an > occasion. .LE .P A clamstress of this enclosedness is cludescing the > hemidormity. .P To arrive at an approximation, a large quantity had been > chromated by a strong feeling. Moreover, that idea sharing was lusated by a > current proposal. Anytime that the final outcomes had been very firmly > unpathesced by not unphilaible reasonable compromises, no serious concerns > might be being sacrated by internal establishments for the basic objectives > in back of a full utili zation. .P As a consequence of the fact that a > total effect might vacate an easily situational beneficial assistance, the > apparent provisioning being effectuated by a continuing difference can have > protenesced a realization of an underly ing purpose. A different doubtful > important outcome is cludated by a capkin. A rationale had fortated > attachments. Moreover, this assumption had nilcoresced the continuing > study. .P .H 1 "An Easily Added Basic Assumption Being Joined Together With > A Concept Stage" There is not an impediment which neoated a restriction, > therefore. A couple utilizations could morsate a great similarity at > considerable difficulties, but an input is primescing the concept > activities, and a growing importance was hemicisesced by that beneficial > assistance. In the same connection, these extremenesses are rather usefully > ultralucesced by directions. .SG .NS 0 C. R. Glitch S. A. Hobble R. S. Limn > M. Shayegan > >> .NE > >> > >> Ed Bradford, Ph.D. Physics, retired from IBM > >> BTL 1976-1983 > >> > >> -- > >> Advice is judged by results, not by intentions. > >> Cicero > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robpike at gmail.com Sat Mar 21 11:48:32 2020 From: robpike at gmail.com (Rob Pike) Date: Sat, 21 Mar 2020 12:48:32 +1100 Subject: [TUHS] I made a tar tape In-Reply-To: References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> Message-ID: #2 actually came from bob, not hardin, it was a safety poster that went up at ITT in Phoenix: KEEP YOUR HANDS OUT OF TIGHT PLACES. REMEMBER YOU ONLY HAVE TWO. -rob On Sat, Mar 21, 2020 at 12:14 PM Ken Thompson wrote: > > two hardin's sprint to mind. > > 1. phonetix > 2. dont put your hands in tight places, > remember you only have two. > > > On Fri, Mar 20, 2020 at 5:29 PM Rob Pike wrote: >> >> Anemometer cur chiffon. >> >> See if that rings any bells. >> >> -rob >> >> On Sat, Mar 21, 2020 at 11:17 AM John P. Linderman wrote: >> > >> > If you enjoyed that, try this, apparently done in 1940. >> > >> > Ron and Bob were something else. I recall Bob "running for DMTS" by handing out dimes in the Murray Hill cafeteria. >> > >> > On Fri, Mar 20, 2020 at 5:28 PM Mary Ann Horton wrote: >> >> >> >> I remember Ron! He was quite a character. And I remember festoon. >> >> >> >> Was it he, or Bob Flandrena, that gave us the holiday poetical and phonetical treat: >> >> >> >> Tweeze Denied Beef Workers Isthmus >> >> >> >> (Say it out loud fast, with a pause in the middle of "Workers", to get it). >> >> >> >> I think I have it on paper somewhere, but no soft copy. >> >> >> >> Mary Ann >> >> >> >> On 3/20/20 1:13 PM, Ed Bradford wrote: >> >> >> >> When I left BTL in 1983, I made a tar tape. A number of years later I translated the tape into a file. Only recently have I wandered through it. >> >> >> >> I don't know how many people remember Ron Hardin in the Columbus BTL location. He was one of the smartest guys I ever met. There are lot of Ron Hardin stories. One of his creations (as far as I know he authored it) was a program to create Memorandums For File -- technical memorandums. My tar tape scooped up festoon. To this day it compiles and runs happily on Windows 10. It was written in 1978 or thereabouts. Here is an example output: >> >> >> >> bin$ festoon.exe .TL No Worthynesses .AU "C. C. Festoon" CCF Headquarters 1584734291 .AS A restriction had been being amicated by a convenience at the inclusion. .AE .MT "MEMORANDUM FOR COAT LOCKER" .hy 1 On this occasion, no team responsibilities could have polyesced a renewed emphasis. A friction had penated an activation. At the present moment in time, an undue number of good progresses being collected together with the populations were being proportionately fideated by the fact that there was a data stream which was transenniesced by an issuance being joined together with these team re sponsibilities, because natural basises have been veriating a partitioning. The supplementary work should be conclusively quinquepolyated by a well defined interfacing. A sophisticatedness by a schedule is operated by a nature in conflict with a correspondence under some serious discussi ons. It is within the realm of possibility that the effectiveness had vicfacesced a schedule, but there was not a necessary background information which is being testesced by a strong interest, and a statistical accuracy was tempoesced by the preparation. It should be noted that a joint partnership very repeatedly aidioated this publication of a centralized organization. Due to the fact that there is a simplification which simply enniesced a process, a new technology is fluxesced from monorogatities. It is of the utmost importance that an insurance could be putated by an assumption. A major advance centered about a deficiency octocessates an important outcome. .P An effectation would extramicroate to the situation. A complete revision gravated a direction. Inasmuch as there was not a potential usefulness that cedeates by the timely delivery, a consideration centered around a technique was monofortated by an integration: .BL .LI There is a not unclear meterdom which had risiesced an occasion. .LE .P A clamstress of this enclosedness is cludescing the hemidormity. .P To arrive at an approximation, a large quantity had been chromated by a strong feeling. Moreover, that idea sharing was lusated by a current proposal. Anytime that the final outcomes had been very firmly unpathesced by not unphilaible reasonable compromises, no serious concerns might be being sacrated by internal establishments for the basic objectives in back of a full utili zation. .P As a consequence of the fact that a total effect might vacate an easily situational beneficial assistance, the apparent provisioning being effectuated by a continuing difference can have protenesced a realization of an underly ing purpose. A different doubtful important outcome is cludated by a capkin. A rationale had fortated attachments. Moreover, this assumption had nilcoresced the continuing study. .P .H 1 "An Easily Added Basic Assumption Being Joined Together With A Concept Stage" There is not an impediment which neoated a restriction, therefore. A couple utilizations could morsate a great similarity at considerable difficulties, but an input is primescing the concept activities, and a growing importance was hemicisesced by that beneficial assistance. In the same connection, these extremenesses are rather usefully ultralucesced by directions. .SG .NS 0 C. R. Glitch S. A. Hobble R. S. Limn M. Shayegan >> >> .NE >> >> >> >> Ed Bradford, Ph.D. Physics, retired from IBM >> >> BTL 1976-1983 >> >> >> >> -- >> >> Advice is judged by results, not by intentions. >> >> Cicero >> >> From dave at horsfall.org Sat Mar 21 14:03:51 2020 From: dave at horsfall.org (Dave Horsfall) Date: Sat, 21 Mar 2020 15:03:51 +1100 (EST) Subject: [TUHS] I made a tar tape In-Reply-To: References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> Message-ID: On Sat, 21 Mar 2020, Rob Pike wrote: > I believe Festoon was covered in a Sci Am article by Dewdney. It > included Mark V Shaney, some computer-generated poetry, and other > horrors, including computer-generated Baudrillard. Ahhh... Mark V Shaney... I knew Bruce Ellis, and I wheedled the source code out of him (with the proviso that I never use it on Usenet) in order to use it on a victim on another medium (Amateur packet radio). Skilfully forging his callsign (and the path), his colleagues thought that he was drunk :-) Source code now lost, dammit... -- Dave From wobblygong at gmail.com Sat Mar 21 14:59:50 2020 From: wobblygong at gmail.com (Wesley Parish) Date: Sat, 21 Mar 2020 17:59:50 +1300 Subject: [TUHS] The most surprising Unix programs In-Reply-To: <20200320145257.GD10251@mcvoy.com> References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> <20200320145257.GD10251@mcvoy.com> Message-ID: In a word: ratios. Each note is in a ratio to each other note being played, either after, while, or after it is being played. It's called the harmonic or overtone series, and if you've ever looked at a half-way decent brass instrument method/tutor, you'll have seen it in all its glory. The nicer-sounding ratios are the ones usually used - the octaves, the fifths, the fourths and the thirds. The others, the sixths, the sevenths and the seconds, tend to clash more and/or add a mournful tone to the music. https://www.howmusicreallyworks.com/Pages_Chapter_3/3_2.html Chapter 3: How Tones and Overtones REALLY Work gives a non-technical but mathematically sound description: table 4 is worth its weight in platinum. Wesley Parish On 3/21/20, Larry McVoy wrote: > On Fri, Mar 20, 2020 at 10:08:50AM -0400, Richard Salz wrote: >> A cool thing about tools is that you never know how it will be used. >> >> Bach's prelude in C major from Well-Tempered Clavier book I, written in >> pure (GNU) sed: https://github.com/laserbat/bach.sed > > If someone understands how this all works, my less capable brain would > love an explanation. My dad and my kid have told me that classical > music is math but I've never seen it. Seems like this is probably > the example I need. > From ches at cheswick.com Sun Mar 22 02:06:31 2020 From: ches at cheswick.com (William Cheswick) Date: Sat, 21 Mar 2020 12:06:31 -0400 Subject: [TUHS] I made a tar tape In-Reply-To: References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> Message-ID: <257E3A63-C939-45A1-8AD8-FC6FFB4B4E46@cheswick.com> Ron made an insult generator. You can get ten fresh samples at https://cheswick.com/insult It has about 42 bits of entropy, generating a nice password that is probably safe-for-work, unless you work in a very unusual place. A lot of people use cuss words as passwords. ches From dave at horsfall.org Sun Mar 22 06:46:33 2020 From: dave at horsfall.org (Dave Horsfall) Date: Sun, 22 Mar 2020 07:46:33 +1100 (EST) Subject: [TUHS] I made a tar tape In-Reply-To: <257E3A63-C939-45A1-8AD8-FC6FFB4B4E46@cheswick.com> References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> <257E3A63-C939-45A1-8AD8-FC6FFB4B4E46@cheswick.com> Message-ID: On Sat, 21 Mar 2020, William Cheswick wrote: > Ron made an insult generator. You can get ten fresh samples at > > https://cheswick.com/insult Neat! I vaguely remember a similar one posted to Usenet ages ago; ah, those were the days. > It has about 42 bits of entropy, generating a nice password that is > probably safe-for-work, unless you work in a very unusual place. A lot > of people use cuss words as passwords. I read somewhere (Schneier?) that "shocking nonsense" makes for good passphrases; easily remembered, and hard to guess. And yes, I know about that XKCD cartoon... -- Dave From imp at bsdimp.com Sun Mar 22 08:29:00 2020 From: imp at bsdimp.com (Warner Losh) Date: Sat, 21 Mar 2020 16:29:00 -0600 Subject: [TUHS] I made a tar tape In-Reply-To: References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> <257E3A63-C939-45A1-8AD8-FC6FFB4B4E46@cheswick.com> Message-ID: On Sat, Mar 21, 2020 at 2:47 PM Dave Horsfall wrote: > On Sat, 21 Mar 2020, William Cheswick wrote: > > > Ron made an insult generator. You can get ten fresh samples at > > > > https://cheswick.com/insult > > Neat! I vaguely remember a similar one posted to Usenet ages ago; ah, > those were the days. > > > It has about 42 bits of entropy, generating a nice password that is > > probably safe-for-work, unless you work in a very unusual place. A lot > > of people use cuss words as passwords. > > I read somewhere (Schneier?) that "shocking nonsense" makes for good > passphrases; easily remembered, and hard to guess. And yes, I know about > that XKCD cartoon... > "play city carrot is a toy store in tokyo" was my pass phrase more than a decade ago.... Not quite nonsense, though, because the Japanese like to use English words for how they look or sound rather than what they mean. Much like people here choose tattoos for how they look, not quite understanding they just got "katchup queen gerbil" emblazoned on their lower back... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From usotsuki at buric.co Sun Mar 22 09:01:56 2020 From: usotsuki at buric.co (Steve Nickolas) Date: Sat, 21 Mar 2020 19:01:56 -0400 (EDT) Subject: [TUHS] I made a tar tape In-Reply-To: References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> <257E3A63-C939-45A1-8AD8-FC6FFB4B4E46@cheswick.com> Message-ID: On Sun, 22 Mar 2020, Dave Horsfall wrote: > On Sat, 21 Mar 2020, William Cheswick wrote: > >> Ron made an insult generator. You can get ten fresh samples at >> >> https://cheswick.com/insult > > Neat! I vaguely remember a similar one posted to Usenet ages ago; ah, those > were the days. > >> It has about 42 bits of entropy, generating a nice password that is >> probably safe-for-work, unless you work in a very unusual place. A lot of >> people use cuss words as passwords. > > I read somewhere (Schneier?) that "shocking nonsense" makes for good > passphrases; easily remembered, and hard to guess. And yes, I know about > that XKCD cartoon... > > -- Dave > I've used non-English words in my passwords, as well as spoonerisms. -uso. From egbegb2 at gmail.com Sun Mar 22 18:25:05 2020 From: egbegb2 at gmail.com (Ed Bradford) Date: Sun, 22 Mar 2020 03:25:05 -0500 Subject: [TUHS] I made a tar tape In-Reply-To: References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> Message-ID: For those interested, the source to festoon can be found here and I cannot be blamed for any copyright infringement. Y'all ought to read Ron's code. Relax with a bottle of Blantons nearby and enjoy. https://web.archive.org/web/20060821141209/http://www.tribug.org/pub/tuhs/Applications/Festoon/ On Fri, Mar 20, 2020 at 5:50 PM Rob Pike wrote: > I believe Festoon was covered in a Sci Am article by Dewdney. It > included Mark V Shaney, some computer-generated poetry, and other > horrors, including computer-generated Baudrillard. > > -rob > > On Sat, Mar 21, 2020 at 9:05 AM Mike Markowski > wrote: > > > > That took me a few tries - love it! :-) - Mike Markowski > > > > On 3/20/20 5:26 PM, Mary Ann Horton wrote: > > > I remember Ron! He was quite a character. And I remember festoon. > > > > > > Was it he, or Bob Flandrena, that gave us the holiday poetical and > > > phonetical treat: > > > > > > Tweeze Denied Beef Workers Isthmus > > > > > > (Say it out loud fast, with a pause in the middle of "Workers", to get > it). > > > > > > I think I have it on paper somewhere, but no soft copy. > > > > > > Mary Ann > -- Advice is judged by results, not by intentions. Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Mon Mar 23 01:58:29 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 22 Mar 2020 09:58:29 -0600 Subject: [TUHS] execl and execv, first appearance in unix Message-ID: Greetings, I was reviewing a manual page update and came across an ambiguous answer to a question that came up, so I thought I'd ask here. execl and execv first appear in our extant unix man pages in V4. The v3 and v2 man pages don't have this listed at all. Case closed, right? It appeared in V4. However, the Dennis_v1/unix72 tree has execl.s and execv.s in them. Diving back into the history on Warren's github account, https://github.com/DoctorWkt/unix-jun72/tree/master/src says they come from: "The files in lib/ come from the libc.sa file which is on the last1120c.tar.gz tap(I) tape image, also at the same URL, and form the C library for the above compiler." and "from a working C compiler for 2nd Edition UNIX." which suggests that it may have been in V2 or maybe even V1. It's first use in Unix appears to be V5, but the extant pre-v5 code is so fragmentary it's hard to know for sure. It's not mentioned in section II or section III of v1, v2 or v3. Does anybody know for sure, or can provide more insight into the last1120c.tar.gz file to help disambiguate? Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From dot at dotat.at Mon Mar 23 04:05:04 2020 From: dot at dotat.at (Tony Finch) Date: Sun, 22 Mar 2020 18:05:04 +0000 Subject: [TUHS] The most surprising Unix programs In-Reply-To: References: <20200320140308.4FBBB18C073@mercury.lcs.mit.edu> Message-ID: Dave Horsfall wrote: > > I have the Towers of Hanoi written in sendmail.cf, meaning that Sendmail is > Turing-complete :-) I can't remember who wrote it. Jonathan Chin: http://www.jhnc.org/hanoi/sendmail-cf.html It's really easy to fall into the Turing tar-pit, if you have a loop around regex-based rewriting - and unix mailers often have routing logic that includes features for rewriting an address and re-routing it. Tony. -- f.anthony.n.finch http://dotat.at/ Thames, Dover: East or southeast 5 to 7, occasionally 4 later. Slight or moderate. Fair. Good. From wkt at tuhs.org Mon Mar 23 06:20:11 2020 From: wkt at tuhs.org (Warren Toomey) Date: Mon, 23 Mar 2020 06:20:11 +1000 Subject: [TUHS] execl and execv, first appearance in unix In-Reply-To: References: Message-ID: <20200322202011.GA32609@minnie.tuhs.org> On Sun, Mar 22, 2020 at 09:58:29AM -0600, Warner Losh wrote: > "from a working C compiler for 2nd Edition UNIX." > Does anybody know for sure, or can provide more insight into the last1120c.tar.g > z file to help disambiguate? Yes, all the details are on Dennis' web page: https://www.bell-labs.com/usr/dmr/www/primevalC.html Cheers! Warren From imp at bsdimp.com Mon Mar 23 06:49:03 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 22 Mar 2020 14:49:03 -0600 Subject: [TUHS] execl and execv, first appearance in unix In-Reply-To: <20200322202011.GA32609@minnie.tuhs.org> References: <20200322202011.GA32609@minnie.tuhs.org> Message-ID: On Sun, Mar 22, 2020 at 2:20 PM Warren Toomey wrote: > On Sun, Mar 22, 2020 at 09:58:29AM -0600, Warner Losh wrote: > > "from a working C compiler for 2nd Edition UNIX." > > > Does anybody know for sure, or can provide more insight into the > last1120c.tar.g > > z file to help disambiguate? > > Yes, all the details are on Dennis' web page: > > https://www.bell-labs.com/usr/dmr/www/primevalC.html Magic historical 8-ball sez: situation unclear, try again. So could be 2nd, 3rd or 4th edition given the date range. Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.winalski at gmail.com Mon Mar 23 07:30:24 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Sun, 22 Mar 2020 17:30:24 -0400 Subject: [TUHS] I made a tar tape In-Reply-To: References: <5eef9b07-7f22-d9c9-78a3-bdb78379cc76@mhorton.net> <257E3A63-C939-45A1-8AD8-FC6FFB4B4E46@cheswick.com> Message-ID: On 3/21/20, Steve Nickolas wrote: > > I've used non-English words in my passwords, as well as spoonerisms. The disk encryption password for my laptop at work was the first line from Jabberwocky with a one-word change and an inserted comma. -Paul W. From wkt at tuhs.org Mon Mar 23 07:44:34 2020 From: wkt at tuhs.org (Warren Toomey) Date: Mon, 23 Mar 2020 07:44:34 +1000 Subject: [TUHS] execl and execv, first appearance in unix In-Reply-To: References: <20200322202011.GA32609@minnie.tuhs.org> Message-ID: <20200322214434.GB13820@minnie.tuhs.org> On Sun, Mar 22, 2020 at 02:49:03PM -0600, Warner Losh wrote: > On Sun, Mar 22, 2020 at 2:20 PM Warren Toomey <[1]wkt at tuhs.org> wrote: > > On Sun, Mar 22, 2020 at 09:58:29AM -0600, Warner Losh wrote: > > "from a working C compiler for 2nd Edition UNIX." > > Does anybody know for sure, or can provide more insight into the > last1120c.tar.g > > z file to help disambiguate? > Yes, all the details are on Dennis' web page: > [2]https://www.bell-labs.com/usr/dmr/www/primevalC.html > > Magic historical 8-ball sez: situation unclear, try again. > So could be 2nd, 3rd or 4th edition given the date range. Someone from the Labs here on the list might be able to clarify. I'm sure I heard that they start with the PDP-11/20 and got an 11/45 around the time of Second Edition. So the "last1120c" was the C compiler around the time of Second Edition. Cheers, Warren From imp at bsdimp.com Mon Mar 23 08:03:20 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 22 Mar 2020 16:03:20 -0600 Subject: [TUHS] execl and execv, first appearance in unix In-Reply-To: <20200322214434.GB13820@minnie.tuhs.org> References: <20200322202011.GA32609@minnie.tuhs.org> <20200322214434.GB13820@minnie.tuhs.org> Message-ID: Hey Warren, On Sun, Mar 22, 2020 at 3:44 PM Warren Toomey wrote: > On Sun, Mar 22, 2020 at 02:49:03PM -0600, Warner Losh wrote: > > On Sun, Mar 22, 2020 at 2:20 PM Warren Toomey <[1]wkt at tuhs.org> > wrote: > > > > On Sun, Mar 22, 2020 at 09:58:29AM -0600, Warner Losh wrote: > > > "from a working C compiler for 2nd Edition UNIX." > > > Does anybody know for sure, or can provide more insight into the > > last1120c.tar.g > > > z file to help disambiguate? > > Yes, all the details are on Dennis' web page: > > [2]https://www.bell-labs.com/usr/dmr/www/primevalC.html > > > > Magic historical 8-ball sez: situation unclear, try again. > > So could be 2nd, 3rd or 4th edition given the date range. > > Someone from the Labs here on the list might be able to clarify. I'm sure > I heard that they start with the PDP-11/20 and got an 11/45 around the time > of Second Edition. So the "last1120c" was the C compiler around the time of > Second Edition. > According to my history of early unix talk, they started to transition to 11/45 in the 2nd edition, and had completed the rewrite by the 4th edition. The 2nd and 3rd editions had, at least according to their intros, some support for the 11/20, but it is ambiguous now that I re-read it: 2nd edition (June '72) "Also, not all installations have the latest version of all the software. In particular, the assembler and loader have just undergone major reorganizations in anticipation of a UNIX for the PDP-11/45" 3rd Edition goes farther (Mar '73): "In particular, any system which uses a PDP-11/20 processor will not include all the software described herein, nor will the software behave in the same way. The second, or even the first edition of this manual is likely to be more appropriate." By the 4th edition (Nov '73), The PDP-11/20 is gone, not even referenced.... "In particular, the information in this manual applies only to UNIX systems which operate under the C language versions of the system. Installations which use older versions of UNIX will find earlier editions of this manual more appropriate to their situation." So the outward messaging of the manual, at least, was "We're running away from the 11/20, and you should too if you want to run the latest code." But it's not clear to me, since we don't have good sources from the actual 2nd or 3rd edition to trace where this came in, or if the 11/20 lived on a little past the 4th edition and had sources freshly loaded on it sometime after the compiler was written... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From roam at ringlet.net Mon Mar 23 18:49:43 2020 From: roam at ringlet.net (Peter Pentchev) Date: Mon, 23 Mar 2020 10:49:43 +0200 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> Message-ID: <20200323084943.GA341861@straylight.m.ringlet.net> On Mon, Mar 09, 2020 at 05:22:57PM -0600, Grant Taylor via TUHS wrote: > On 3/8/20 9:13 AM, Derek Fawcus wrote: > > Now what would have been useful is a way to have distinct fd's for the > > local read and write end of (e.g.) a TCP socket - such that one > > direction could be closed w/o closing the other. > > I believe that this can be done, now. At least I've read that it's possible > for one end to close (FIN) a TCP connection without the other end also > closing. Thus you end up with the one-way data flow that is still ACKed the > way that TCP does. Yep, in the next sentence Derek mentioned "dispense with shutdown", meaning the shutdown(2) syscall that does exactly that. What he meant was, wouldn't it be nice to be able to do that with close(2) instead? G'luck, Peter -- Peter Pentchev roam@{ringlet.net,debian.org,FreeBSD.org} pp at storpool.com PGP key: http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint 2EE7 A7A5 17FC 124C F115 C354 651E EFB0 2527 DF13 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From egbegb2 at gmail.com Mon Mar 23 20:18:45 2020 From: egbegb2 at gmail.com (Ed Bradford) Date: Mon, 23 Mar 2020 05:18:45 -0500 Subject: [TUHS] Question: stdio - Who invented and ... Message-ID: I've forgotten who created stdio, USG or the research group. Can any of the youthful BTL folks of the 1970's refresh my mind. Given that stdio was invented and, in my opinion at the time, a reasonable and usable standard interface to IO on Unix, I am curious why no standard for networking was developed or proposed and discussed. Sockets just defined a new and very quirky IO interface for Unix based systems. Was any thought given to defining networking using the creat/open/read/write/close/mknod/link/unlink/chmod/chown model of IO in UNIX? Ed Bradford -- Advice is judged by results, not by intentions. Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Mon Mar 23 20:32:34 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Mon, 23 Mar 2020 04:32:34 -0600 Subject: [TUHS] Question: stdio - Who invented and ... In-Reply-To: References: Message-ID: <202003231032.02NAWY4v022713@freefriends.org> Ed Bradford wrote: > I've forgotten who created stdio, USG or the research group. Can any of the > youthful BTL folks of the 1970's refresh my mind. It was part of V7. I think DMR gets most of the credit. > Given that stdio was invented and, in my opinion at the time, a reasonable > and usable standard interface to IO on Unix, I am curious why no standard > for networking was developed or proposed and discussed. Sockets just > defined a new and very quirky IO interface for Unix based systems. > > Was any thought given to defining networking > using the > creat/open/read/write/close/mknod/link/unlink/chmod/chown > > model of IO in UNIX? Much of this has been discussed (to death) already on this list, and fairly recently, too. At that time networking was still a research topic. The Bell Labs folks worked on it, and this is visible in the streams stuff in V8/V9/V10 and early Plan 9, but by that time their work was less influential on the wider Unix community. There were efforts to integrate networking into the Unix file namespace, but it doesn't fit overly cleanly, and such things didn't spread. BSD networking adopted sockets from other, earlier efforts. IMHO less thought was given to "integration with Unix ideas" as opposed to just getting something working and usable, but that's just my opinion based on hindsight. BSD sockets spread because BSD was in the right place at the right time: it ran on the Vax, it also provided paging. Straight research Unix did not support the vax at the time, and people were looking to move to the 32 bit environment. All that plus csh with its interactive features (history, job control) and the vi screen editor (with ed inside it), made BSD (and thus its networking) very popular. Again, all of this is my 2 cents, and there's much more to be found in the list archives. HTH, Arnold From clemc at ccc.com Mon Mar 23 23:46:52 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 23 Mar 2020 09:46:52 -0400 Subject: [TUHS] Question: stdio - Who invented and ... In-Reply-To: <202003231032.02NAWY4v022713@freefriends.org> References: <202003231032.02NAWY4v022713@freefriends.org> Message-ID: On Mon, Mar 23, 2020 at 6:33 AM wrote: > Ed Bradford wrote: > > > I've forgotten who created stdio, USG or the research group. Can any of > the > > youthful BTL folks of the 1970's refresh my mind. > > It was part of V7. I think DMR gets most of the credit. At this risk of putting too fine a point on it, the stdio library was released in the wild before V7 or UNIX/TS *et al.* To answer Ed's question, it came out of Research, but first as part of the typesetter support - *i.e.* 'Typesetter C', which was on V6 and PWB 1.0 [the new troff replacement was being written by Brian] . The C Language and associated libraries in the 'Typesetter C' release maps to the compiler described in the original K&R book. Dennis explains this in his paper: The Development of the C Language . *"Lesk wrote a `portable I/O package' [Lesk 72] that was later reworked to become the C `standard I/O' routines. In 1978 Brian Kernighan and I published The C Programming Language [Kernighan 78]. "* I have the Lesk paper, as PDF (which I have not idea where I obtained). I did a quick google search and could not find it for download, so if you are interested, send me an e-mail offline and I'll pass you a copy. I've forgotten when 'enum' and 'void' got added (which are not in the white book - Steve Johnson or Doug may remember). But, I think they were in the V7 compiler, and not Typesetter C. Clem -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Mon Mar 23 23:59:19 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Mon, 23 Mar 2020 07:59:19 -0600 Subject: [TUHS] Question: stdio - Who invented and ... In-Reply-To: References: <202003231032.02NAWY4v022713@freefriends.org> Message-ID: <202003231359.02NDxJU6030344@freefriends.org> Clem Cole wrote: > To answer Ed's question, it came out of Research, but first as part of the > typesetter support - *i.e.* 'Typesetter C', which was on V6 and PWB > 1.0 [the new troff replacement was being written by Brian] . The troff replacement was somewhat after V7. The troff that shipped with V7 was still only for the CAT. See https://www.cs.princeton.edu/~bwk/202/summer.reconstructed.pdf which is dated January 1980. Arnold From jaapna at xs4all.nl Tue Mar 24 00:24:05 2020 From: jaapna at xs4all.nl (Jaap Akkerhuis) Date: Mon, 23 Mar 2020 15:24:05 +0100 Subject: [TUHS] Question: stdio - Who invented and ... In-Reply-To: <202003231359.02NDxJU6030344@freefriends.org> References: <202003231032.02NAWY4v022713@freefriends.org> <202003231359.02NDxJU6030344@freefriends.org> Message-ID: <01040037-4D96-4DCB-8E40-B418E424C0F5@xs4all.nl> > On Mar 23, 2020, at 14:59, arnold at skeeve.com wrote: > > Clem Cole wrote: > >> To answer Ed's question, it came out of Research, but first as part of the >> typesetter support - *i.e.* 'Typesetter C', which was on V6 and PWB >> 1.0 [the new troff replacement was being written by Brian] . > > The troff replacement was somewhat after V7. The troff that shipped with V7 > was still only for the CAT. Yes, "Typesetter C" was to accommodate the C Version of troff written by Ossana and I checked the source, it doesn't use stdio. jaap -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 267 bytes Desc: Message signed with OpenPGP URL: From clemc at ccc.com Tue Mar 24 00:45:48 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 23 Mar 2020 10:45:48 -0400 Subject: [TUHS] Question: stdio - Who invented and ... In-Reply-To: <01040037-4D96-4DCB-8E40-B418E424C0F5@xs4all.nl> References: <202003231032.02NAWY4v022713@freefriends.org> <202003231359.02NDxJU6030344@freefriends.org> <01040037-4D96-4DCB-8E40-B418E424C0F5@xs4all.nl> Message-ID: On Mon, Mar 23, 2020 at 10:24 AM Jaap Akkerhuis wrote: > Yes, "Typesetter C" was to accommodate the C Version of troff written > by Ossana and I checked the source, it doesn't use stdio. > Interesting. The compiler described in K&R was a v6 compiler -- we had it at CMU before we had V7. I thought we got it from the Typesetter C release. But Ted may have brought even before that. Brian K. might have also. He was Brian Ried's thesis committee and was coming to CMU reasonably often in 77/78 timeframe - when Reid was writing Scribe and Brian ditroff. I'm pretty sure we got some stuff via that channel too, as I was the person moving some of that around campus and its how I first met Brian. I admit that I might have some void in my memory, but I do remember part of this pretty well. Because CS had the 11/40E with writable microcode (they had been developed for C.mmp). The CS PDP-11's running UNIX (IUS and SUS) had CSV/CRET instructions (as well as a couple of others for BLISS and support capabilities, IIRC). Someone (Paul Rubin I think), had modified the original V6 compiler to generate CSV/CRET. That meant that binaries from CS would not run on the 11/34's which most of the UNIX box on campus had by then become. Ted and I were in EE and there was a small battle as to which systems were more current. Since Ted worked officially for USG, he had a direct line to much of there stuff. Ted had xerographic proofs of the book when it was being reviewed BTW. When CMU got the new Ritchie compiler and I had the task of rippling it out to all the sites in EE, Mellon Institute, et al. CS did not pick it up originally because it did not have the CSAV/CRET changes. CMU did not get V7 until late winter '78/early spring '79 around the time I was leaving. So, we definitely had a Ritchie based K&R compiler with stdio. I had thought that come to us with Typesetter C, but it sounds like it had a different path. FWIW: that would actually be the compiler sources I started with at Tektronix 6 months later, when I started to retarget it for what would become the 68000 (it was not yet released and was an experimental chip), as I had brought the CMU UNIX systems with me. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich.salz at gmail.com Tue Mar 24 04:47:16 2020 From: rich.salz at gmail.com (Richard Salz) Date: Mon, 23 Mar 2020 14:47:16 -0400 Subject: [TUHS] Question: stdio - Who invented and ... In-Reply-To: <01040037-4D96-4DCB-8E40-B418E424C0F5@xs4all.nl> References: <202003231032.02NAWY4v022713@freefriends.org> <202003231359.02NDxJU6030344@freefriends.org> <01040037-4D96-4DCB-8E40-B418E424C0F5@xs4all.nl> Message-ID: > Yes, "Typesetter C" was to accommodate the C Version of troff written > by Ossana and I checked the source, it doesn't use stdio. > I thought Ossana's troff was written in assembler and translitered back into C after he died. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaapna at xs4all.nl Tue Mar 24 05:22:36 2020 From: jaapna at xs4all.nl (Jaap Akkerhuis) Date: Mon, 23 Mar 2020 20:22:36 +0100 Subject: [TUHS] Question: stdio - Who invented and ... In-Reply-To: References: <202003231032.02NAWY4v022713@freefriends.org> <202003231359.02NDxJU6030344@freefriends.org> <01040037-4D96-4DCB-8E40-B418E424C0F5@xs4all.nl> Message-ID: <892AF29A-10FC-473D-BB3F-4FAEF20719C2@xs4all.nl> > On Mar 23, 2020, at 19:47, Richard Salz wrote: > > > Yes, "Typesetter C" was to accommodate the C Version of troff written > by Ossana and I checked the source, it doesn't use stdio. > > I thought Ossana's troff was written in assembler and translitered back into C after he died. Nope. I have it from good a good source he did it himself. I seem to remember that the old assembler version had build in constructs for headers and footers and which got radically removed in the C-version. That one gave you the famous trap mechanism etc. Although the basic structure of the assembler version starts out the same it quickly deviates, so it cannot have been turned from as into C. The N/TROFF manual describing the C version stems from 1977. Wikipedia dates the rewrite into C from 1975 (See >). I seem to remember that there where actually two versions of Edition 6 unix. The second one came with typesetter C but my memory is foggy about these details. jaap -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 267 bytes Desc: Message signed with OpenPGP URL: From dave at horsfall.org Tue Mar 24 07:38:38 2020 From: dave at horsfall.org (Dave Horsfall) Date: Tue, 24 Mar 2020 08:38:38 +1100 (EST) Subject: [TUHS] Question: stdio - Who invented and ... In-Reply-To: References: <202003231032.02NAWY4v022713@freefriends.org> Message-ID: On Mon, 23 Mar 2020, Clem Cole wrote: > At this risk of putting too fine a point on it, the stdio library was > released in the wild before V7 or UNIX/TS et al.   Yep; I was running it on V6, post "portable I/O library". I think I grabbed it from PWB. -- Dave From woods at robohack.ca Tue Mar 24 12:02:16 2020 From: woods at robohack.ca (Greg A. Woods) Date: Mon, 23 Mar 2020 19:02:16 -0700 Subject: [TUHS] Question: stdio - Who invented and ... In-Reply-To: References: <202003231032.02NAWY4v022713@freefriends.org> Message-ID: At Mon, 23 Mar 2020 09:46:52 -0400, Clem Cole wrote: Subject: Re: [TUHS] Question: stdio - Who invented and ... > > I've forgotten when 'enum' and 'void' got added (which are not in the white > book - Steve Johnson or Doug may remember). But, I think they were in the > V7 compiler, and not Typesetter C. Since I was recently researching these myself: There was an extra page in the 7th Edition manual titled "Recent Changes to C" which described both structure assignment and the enumeration type: https://www.bell-labs.com/usr/dmr/www/cchanges.pdf This paper appears in the UNIX System III "The C Programming Language Reference Manual", but there's no mention of "void" in that manual. On the other hand the UNIX System III PDP-11 compiler mentions "void" (1980). I don't see any mention of "void" in 7th Edition sources. However the version of 'awk' on the v7addenda tape from "12/2/80" has one "(void)" cast. The only mention of "void" in dmr's "The Development of the C Language" paper (from HOPL-II, 1993) seems to be in the "Standardization" section where it's mentioned that it's not described in the first edition of K&R. There's mention in the CSTR#102 paper from Sept. 1981 of the "void" type. The 2.9BSD code uses "void", but the sources I have don't include a copy of the compiler. -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From dfawcus+lists-tuhs at employees.org Tue Mar 24 19:47:10 2020 From: dfawcus+lists-tuhs at employees.org (Derek Fawcus) Date: Tue, 24 Mar 2020 10:47:10 +0100 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: <20200323084943.GA341861@straylight.m.ringlet.net> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> Message-ID: <20200324094710.GB83316@clarinet.employees.org> On Mon, Mar 23, 2020 at 10:49:43AM +0200, Peter Pentchev wrote: > On Mon, Mar 09, 2020 at 05:22:57PM -0600, Grant Taylor via TUHS wrote: > > On 3/8/20 9:13 AM, Derek Fawcus wrote: > > > Now what would have been useful is a way to have distinct fd's for the > > > local read and write end of (e.g.) a TCP socket - such that one > > > direction could be closed w/o closing the other. > > > > I believe that this can be done, now. At least I've read that it's possible > > for one end to close (FIN) a TCP connection without the other end also > > closing. Thus you end up with the one-way data flow that is still ACKed the > > way that TCP does. > > Yep, in the next sentence Derek mentioned "dispense with shutdown", > meaning the shutdown(2) syscall that does exactly that. What he meant > was, wouldn't it be nice to be able to do that with close(2) instead? Quite. One point being that one could fork/exec a program with those fd's attached to stdin/stdout and it could operate as a normal filter, w/o having to understand it was using a socket. (i.e. closing stdout [hence triggering a FIN], while still reading from stdin) Plus various other games achievable by replumbing fd's. DF From gtaylor at tnetconsulting.net Thu Mar 26 09:25:38 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 25 Mar 2020 17:25:38 -0600 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: <20200324094710.GB83316@clarinet.employees.org> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> Message-ID: <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> On 3/24/20 3:47 AM, Derek Fawcus wrote:> One point being that one could fork/exec a program with those fd's > attached to stdin/stdout and it could operate as a normal filter, w/o > having to understand it was using a socket. Hum. This is quite intriguing. Now I'm wondering what sort of mistchif I could get up to doing things with Bash (et al.) supporting /dev/tcp pseudo files. > (i.e. closing stdout [hence triggering a FIN], while still reading > from stdin) I like it. > Plus various other games achievable by replumbing fd's. Ya. There's quite a bit to think about here. -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From rich.salz at gmail.com Thu Mar 26 09:47:48 2020 From: rich.salz at gmail.com (Richard Salz) Date: Wed, 25 Mar 2020 19:47:48 -0400 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> Message-ID: > One point being that one could fork/exec a program with those fd's > attached to stdin/stdout and it could operate as a normal filter, w/ Hasn't that pretty much worked ever since BSD wrote inetd? -------------- next part -------------- An HTML attachment was scrubbed... URL: From gtaylor at tnetconsulting.net Thu Mar 26 10:11:14 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 25 Mar 2020 18:11:14 -0600 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> Message-ID: On 3/25/20 5:47 PM, Richard Salz wrote: > Hasn't that pretty much worked ever since BSD wrote inetd? (x)inetd does allow for the incoming connections (and outgoing replies). But I'm not aware of (x)inetd altering anything for programs initiating new outbound connections. -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From rich.salz at gmail.com Thu Mar 26 10:18:42 2020 From: rich.salz at gmail.com (Richard Salz) Date: Wed, 25 Mar 2020 20:18:42 -0400 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> Message-ID: > (x)inetd does allow for the incoming connections (and outgoing replies). > But I'm not aware of (x)inetd altering anything for programs > initiating new outbound connections. > netcat ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From gtaylor at tnetconsulting.net Thu Mar 26 11:08:35 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 25 Mar 2020 19:08:35 -0600 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> Message-ID: On 3/25/20 6:18 PM, Richard Salz wrote: > netcat ? Sure. netcat, et al., do make it possible to initiate outbound connections. Maybe it's the fog in my brain at the moment, but I don't see how to connect netcat's STDIO to another process cleanly. Maybe I need to think more about the following hypothetical example: · netcat's STDOUT to grep's STDIN · grep's STDOUT to netcat's STDIN Where netcat listens on a port and grep does some useful work over the stream coming through netcat. Thank you for prodding my brain Richard. -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From dot at dotat.at Thu Mar 26 11:20:35 2020 From: dot at dotat.at (Tony Finch) Date: Thu, 26 Mar 2020 01:20:35 +0000 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> Message-ID: So I fooled around making an HTTP request from bash... (printf 'HEAD / HTTP/1.0\r\nHost: dotat.at\r\n\r\n'; cat 1>&2) \ 0<>/dev/tcp/dotat.at/http 1>&0 And I wondered if it would work with ksh too. It does, but there is weirdness. Bash's <> operator follows POSIX, so in my one-liner the 0 is redundant. https://pubs.opengroup.org/onlinepubs/007908799/xcu/chap2.html#tag_001_007_007 However when I look at a ksh man page I find it says https://github.com/att/ast/blob/master/src/cmd/ksh93/sh.1#L3458 <>word Open file word for reading and writing as standard output. I thought POSIX got features like this from ksh so I'm curious that a weird little incompatibilty like this has crept in. (The copy of ksh.1 I have from the CSRG archives lacks <> so I guess that version was ksh88?) Tony. -- f.anthony.n.finch http://dotat.at/ Sole: Northerly 6 or 7 at first in west, otherwise easterly or northeasterly 5 or 6. Moderate or rough, occasionally very rough in west. Showers in west. Good. From dave at horsfall.org Thu Mar 26 12:38:17 2020 From: dave at horsfall.org (Dave Horsfall) Date: Thu, 26 Mar 2020 13:38:17 +1100 (EST) Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> Message-ID: On Wed, 25 Mar 2020, Grant Taylor via TUHS wrote: > · netcat's STDOUT to grep's STDIN > · grep's STDOUT to netcat's STDIN Are you trying to set up a loop of processes or something? I'm not sure if that is even possible, although you can't rule out creative uses of dup2() etc... -- Dave From robpike at gmail.com Thu Mar 26 13:08:41 2020 From: robpike at gmail.com (Rob Pike) Date: Thu, 26 Mar 2020 14:08:41 +1100 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> Message-ID: I can't find it now, but there's a very short rc script that does a modestly realistic telnet client in Plan 9. But you know, that's not Unix. -rob On Thu, Mar 26, 2020 at 1:39 PM Dave Horsfall wrote: > > On Wed, 25 Mar 2020, Grant Taylor via TUHS wrote: > > > · netcat's STDOUT to grep's STDIN > > · grep's STDOUT to netcat's STDIN > > Are you trying to set up a loop of processes or something? I'm not sure > if that is even possible, although you can't rule out creative uses of > dup2() etc... > > -- Dave From ggm at algebras.org Thu Mar 26 13:43:01 2020 From: ggm at algebras.org (George Michaelson) Date: Thu, 26 Mar 2020 13:43:01 +1000 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> Message-ID: Options negotiation and the URG/PUSH always freaked me out. PAD (the X.25 equivalent) was a bugger to work with. From memory, the yorkbox was a forked pair of processes one to read, one to write. it didn't work very well to be honest. I tended to convert to half-duplex mode and construct valid lines of input before sending them (which is not very editor friendly unelss you like Teco, which I didn't since it was complicated. I stuck to SOS and ed) Telnet is pretty much just "read and write for networks" except for the options. Back in the days of the BBN Butterfly, the gethostaddr() table for de interwebz was a linear list, and UCL was at the back of the hosts.txt sort and the time it took the daemon to work out who we were, for a login: prompt, was 1-2 sec close to the 30 second drop-link-he's-dead-jim timer in getty or whatever it was then. Sad. I think we made a lot of drama about read and write for networks. Really, asynchronous communicating processes is a lot of fun. I went to Milners lectures on the calculus of communicating systems, it was also too hard, I lost it. That was also when the real underlying routing (pre BGP) was a push-down list, which dropped our routes because we were the boring british side of things and LRU cache said no. Rebooting the right fuzzball or BBN box generall brought it back. X25 was a good fit for PAD. small packets. Enter the ATM cell size discussion <----here. On Thu, Mar 26, 2020 at 1:09 PM Rob Pike wrote: > > I can't find it now, but there's a very short rc script that does a > modestly realistic telnet client in Plan 9. > > But you know, that's not Unix. > > -rob > > On Thu, Mar 26, 2020 at 1:39 PM Dave Horsfall wrote: > > > > On Wed, 25 Mar 2020, Grant Taylor via TUHS wrote: > > > > > · netcat's STDOUT to grep's STDIN > > > · grep's STDOUT to netcat's STDIN > > > > Are you trying to set up a loop of processes or something? I'm not sure > > if that is even possible, although you can't rule out creative uses of > > dup2() etc... > > > > -- Dave From gtaylor at tnetconsulting.net Thu Mar 26 14:11:03 2020 From: gtaylor at tnetconsulting.net (Grant Taylor) Date: Wed, 25 Mar 2020 22:11:03 -0600 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> Message-ID: <25956003-d4df-36ab-fd89-37c90bf40a13@spamtrap.tnetconsulting.net> On 3/25/20 8:38 PM, Dave Horsfall wrote: > Are you trying to set up a loop of processes or something? Nope. I'll try another way +------------------+ +-------------+ --(TCP)-->+ socket stdout +---+ stdin | | nc | | grep | <--(TCP)--| socket stdin +---+ stdout | +------------------+ +-------------+ This example is back to the functionality that (x)inetd would provide. The idea being that grep* would act on the data that came in the TCP connection and send the matching lines out the same TCP connection. *grep is just a hypothetical example here. > I'm not sure if that is even possible, although you can't rule out > creative uses of dup2() etc... ~chuckle~ -- Grant. . . . unix || die -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4013 bytes Desc: S/MIME Cryptographic Signature URL: From dfawcus+lists-tuhs at employees.org Fri Mar 27 20:51:12 2020 From: dfawcus+lists-tuhs at employees.org (Derek Fawcus) Date: Fri, 27 Mar 2020 11:51:12 +0100 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: References: <20200306224431.D226C18C080@mercury.lcs.mit.edu> <3D1DBF45-AE50-4027-8AAA-6C1D97D28D4D@planet.nl> <20200307163935.GA57521@clarinet.employees.org> <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> Message-ID: <20200327105112.GA11907@clarinet.employees.org> On Wed, Mar 25, 2020 at 07:47:48PM -0400, Richard Salz wrote: > > One point being that one could fork/exec a program with those fd's > > attached to stdin/stdout and it could operate as a normal filter, w/ > > Hasn't that pretty much worked ever since BSD wrote inetd? The socket fd is dup'ed and then applied to stdin & stdout for the child process. So if say the child closes stdout, stdin still has the underlying 'file' open in both read and write mode, hence the reader at the far end will not receive any form of 'close' notification (e.g. EOF on read). If a socket fd could be split (or partially dup'ed) in to new read-only and write-only fds, and the original fd then closed, that 'issue' would go away. Obviously it hasn't been much of an issue... DF From wkt at tuhs.org Sat Mar 28 19:49:18 2020 From: wkt at tuhs.org (Warren Toomey) Date: Sat, 28 Mar 2020 19:49:18 +1000 Subject: [TUHS] Article on Unix pipes history Message-ID: https://toroid.org/unix-pipe-implementation Quite good. Cheers, Warren -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roam at ringlet.net Sat Mar 28 21:12:34 2020 From: roam at ringlet.net (Peter Pentchev) Date: Sat, 28 Mar 2020 13:12:34 +0200 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: References: <20200308151332.GA70333@clarinet.employees.org> <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> Message-ID: <20200328111234.GA342169@straylight.m.ringlet.net> On Thu, Mar 26, 2020 at 01:38:17PM +1100, Dave Horsfall wrote: > On Wed, 25 Mar 2020, Grant Taylor via TUHS wrote: > > > · netcat's STDOUT to grep's STDIN > > · grep's STDOUT to netcat's STDIN > > Are you trying to set up a loop of processes or something? I'm not sure if > that is even possible, although you can't rule out creative uses of dup2() > etc... This can't really be done with netcat, but it's quite easy to do with socat; here's an example with a trivial program that reads lines from its standard input and writes a single line to its standard output: [roam at straylight ~]$ socat -v tcp4:nimbus.fccf.net:25 exec:./heysmtp.py > 2020/03/28 13:09:04.005497 length=48 from=0 to=47 220 nimbus.fccf.net ESMTP Postfix (Debian/GNU)\r < 2020/03/28 13:09:04.018931 length=6 from=0 to=5 QUIT\r > 2020/03/28 13:09:04.035387 length=15 from=48 to=62 221 2.0.0 Bye\r [roam at straylight ~]$ All the output was actually from socat because of the "-v" option specified. G'luck, Peter -- Peter Pentchev roam@{ringlet.net,debian.org,FreeBSD.org} pp at storpool.com PGP key: http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint 2EE7 A7A5 17FC 124C F115 C354 651E EFB0 2527 DF13 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From roam at ringlet.net Sat Mar 28 21:14:28 2020 From: roam at ringlet.net (Peter Pentchev) Date: Sat, 28 Mar 2020 13:14:28 +0200 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: <20200328111234.GA342169@straylight.m.ringlet.net> References: <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> <20200328111234.GA342169@straylight.m.ringlet.net> Message-ID: <20200328111428.GA1431416@straylight.m.ringlet.net> On Sat, Mar 28, 2020 at 01:12:35PM +0200, Peter Pentchev wrote: > On Thu, Mar 26, 2020 at 01:38:17PM +1100, Dave Horsfall wrote: > > On Wed, 25 Mar 2020, Grant Taylor via TUHS wrote: > > > > > · netcat's STDOUT to grep's STDIN > > > · grep's STDOUT to netcat's STDIN > > > > Are you trying to set up a loop of processes or something? I'm not sure if > > that is even possible, although you can't rule out creative uses of dup2() > > etc... > > This can't really be done with netcat, but it's quite easy to do with > socat; here's an example with a trivial program that reads lines from > its standard input and writes a single line to its standard output: > > [roam at straylight ~]$ socat -v tcp4:nimbus.fccf.net:25 exec:./heysmtp.py > > 2020/03/28 13:09:04.005497 length=48 from=0 to=47 > 220 nimbus.fccf.net ESMTP Postfix (Debian/GNU)\r > < 2020/03/28 13:09:04.018931 length=6 from=0 to=5 > QUIT\r > > 2020/03/28 13:09:04.035387 length=15 from=48 to=62 > 221 2.0.0 Bye\r > [roam at straylight ~]$ > > All the output was actually from socat because of the "-v" option > specified. ...but, of course, this is still not what Derek was talking about earlier - there is no separation of the file descriptors connected to the socket: closing the stdout one would not result in a FIN being sent along the line. G'luck, Peter -- Peter Pentchev roam@{ringlet.net,debian.org,FreeBSD.org} pp at storpool.com PGP key: http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint 2EE7 A7A5 17FC 124C F115 C354 651E EFB0 2527 DF13 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From steffen at sdaoden.eu Sun Mar 29 02:03:55 2020 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Sat, 28 Mar 2020 17:03:55 +0100 Subject: [TUHS] sockets (was Re: First appearance of named pipes) In-Reply-To: <20200328111428.GA1431416@straylight.m.ringlet.net> References: <5dbf2153-5993-242a-5ba9-54b49a38569e@spamtrap.tnetconsulting.net> <20200323084943.GA341861@straylight.m.ringlet.net> <20200324094710.GB83316@clarinet.employees.org> <127f7c5c-a607-8f5b-1760-5a308c279836@tnetconsulting.net> <20200328111234.GA342169@straylight.m.ringlet.net> <20200328111428.GA1431416@straylight.m.ringlet.net> Message-ID: <20200328160355.NCc9a%steffen@sdaoden.eu> Peter Pentchev wrote in <20200328111428.GA1431416 at straylight.m.ringlet.net>: |On Sat, Mar 28, 2020 at 01:12:35PM +0200, Peter Pentchev wrote: |> On Thu, Mar 26, 2020 at 01:38:17PM +1100, Dave Horsfall wrote: |>> On Wed, 25 Mar 2020, Grant Taylor via TUHS wrote: |>> |>>> · netcat's STDOUT to grep's STDIN |>>> · grep's STDOUT to netcat's STDIN |>> |>> Are you trying to set up a loop of processes or something? I'm \ |>> not sure if |>> that is even possible, although you can't rule out creative uses \ |>> of dup2() |>> etc... |> |> This can't really be done with netcat, but it's quite easy to do with |> socat; here's an example with a trivial program that reads lines from |> its standard input and writes a single line to its standard output: |> |> [roam at straylight ~]$ socat -v tcp4:nimbus.fccf.net:25 exec:./heysmtp.\ |> py ... perl(1) has IPC::Open2 for that: use IPC::Open2; # We use `csop' for hashing my $MAILX = 'LC_ALL=C s-nail -#:/'; sub hash_em{ die "hash_em: open: $^E" unless my $pid = open2 *RFD, *WFD, $MAILX; foreach my $e (@ENTS){ print WFD "csop hash32 $e->{name}\n"; my $h = ; chomp $h; $e->{hash} = $h } print WFD "x\n"; waitpid $pid, 0; } hash_em() |...but, of course, this is still not what Derek was talking about |earlier - there is no separation of the file descriptors connected to |the socket: closing the stdout one would not result in a FIN being sent |along the line. Just wanted to add earlier in the thread that on some systems shutdown(2) equals close(2). At least it was like that on Mac OS X (by then) Snow Leopard. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From jnc at mercury.lcs.mit.edu Sun Mar 29 03:43:09 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Sat, 28 Mar 2020 13:43:09 -0400 (EDT) Subject: [TUHS] Article on Unix pipes history Message-ID: <20200328174309.DD9CE18C0A1@mercury.lcs.mit.edu> > From: Warren Toomey > Quite good Yes. The second part of: "Sixth Edition sources are much more widely available than earlier versions, thanks largely to the Lions book" is I believe incorrect, though; I reckon it's because V6 was so widely distributed, both inside and outside Bell. Many more copies -> higher probability of retention. Also, from 'Source code listing for the Lions' Commentary in PDF and PostScript': "in 1988 I discovered an old 9-track tape being discarded of a PDP11 backup. It was hard to determine what it was running, but it did have an intact /usr/src/ tree of which most of the files were timesamped 1979, even at that time it seemed ancient. So it was either 7th edition or a derivative like PWB, which I believe it was. Do you still have the tape, or its contents? Noel From pnr at planet.nl Sun Mar 29 22:27:58 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sun, 29 Mar 2020 14:27:58 +0200 Subject: [TUHS] 8th Edition timeline Message-ID: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> Here is a question for the old hands from the Labs, I’m trying to get the timeline of some development steps right. The two main things are: when did the 4.1 merge take place, and when were ‘streams’ added? Going by file dates, the surviving 8th edition source appears to be from 1985. I can see that a lot of files in /usr/include did not change after Jan 1982 (e.g. nlist.h). This suggests that early in 1982 the merge between 4.1 code and 32V code took place, to create the foundation for further development (“proto 8th edition”, so to speak). Similarly, there are a dozen or so files in the kernel that all have a file date of November 1982. The most interesting one of these is “dtline.c”, a character mode Datakit driver: it uses ‘streams’. This suggests that there was a further code merge late in 1982 and implies that ‘streams’ were developed prior to that date. From the S/F-Unix papers it seems that ‘streams’ did not exist in 1981, at least they are not mentioned in an otherwise comprehensive set of papers. On the other hand, the S/F-Unix work was done in the Exploratory group, not the Research group: maybe it was inappropriate to mention. All in all, my hypotheses would be that: - the 32V/4.1 merge took place early in 1982 - ‘streams’ were developed in 1982 on 32V (maybe also V7) systems - a further merge took place late in 1982 that combined the new base with latest developments Does that sound correct, or was it all different? Related is the question when the "file system switch" was added. It must have been later than 1981 and before 1985, but I have not been able to pinpoint it further. Paul From arnold at skeeve.com Mon Mar 30 00:04:39 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sun, 29 Mar 2020 08:04:39 -0600 Subject: [TUHS] 8th Edition timeline In-Reply-To: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> Message-ID: <202003291404.02TE4dAI022916@freefriends.org> Paul Ruizendaal wrote: > Related is the question when the "file system switch" was added. It must > have been later than 1981 and before 1985, but I have not been able to > pinpoint it further. IIRC there was a "paper" (only an abstract) on the file system switch published in a USENIX conference proceedings. That woud help trace it down. Peter Weinberger, who did it, is at Google; you could ask him directly, as well. Arnold From pnr at planet.nl Mon Mar 30 04:12:32 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sun, 29 Mar 2020 20:12:32 +0200 Subject: [TUHS] 8th Edition timeline In-Reply-To: <202003291404.02TE4dAI022916@freefriends.org> References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> Message-ID: <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> On 29 Mar 2020, at 16:04, arnold at skeeve.com wrote: > > Paul Ruizendaal wrote: > >> Related is the question when the "file system switch" was added. It must >> have been later than 1981 and before 1985, but I have not been able to >> pinpoint it further. > > IIRC there was a "paper" (only an abstract) on the file system > switch published in a USENIX conference proceedings. That woud help > trace it down. I have that paper (“The Unix 8th Edition Network File System”), it was presented at a March 1985 ACM conference. However, there are indications that the roots of the file system switch existed earlier, possibly much earlier. I think Doug McIlroy once described 1973 as a pivotal year for Unix, with many concepts devised that would blossom in the following 3-5 years. I’m increasingly tempted to think that Summer ’81 - Summer ’82 was a similarly pivotal year. > Peter Weinberger, who did it, is at Google; you could ask him > directly, as well. That is a good idea. If someone has the email address I’d appreciate an off list message. Paul From robpike at gmail.com Mon Mar 30 07:48:20 2020 From: robpike at gmail.com (Rob Pike) Date: Mon, 30 Mar 2020 08:48:20 +1100 Subject: [TUHS] 8th Edition timeline In-Reply-To: <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> Message-ID: I have a dog in the fight, having joined in June 1980, but that is not a coincidence. The period of 1980-1982 was a big one for 127 (soon 1127) as they were finally given the chance to grow, and I was one of the lucky early hires in that burst. New blood brought in new ideas and things happened fast. It was also the time of the VAX; the center's 11/780 arrived in late 1980 I think, maybe early 1981. Our first experiments with graphical terminals spanned 1980 to early 1981, using Greg Chesson's mux, but by late 1981 we were using Dennis's streams (only STREAMS when they went to USG) and the select system call, which was by then running in a merged Berkeley/Research Unix that eventually became the Eighth Edition. My notebooks can probably lock down a lot of this as I was a prolific note-taker back then, when they still made paper. -rob On Mon, Mar 30, 2020 at 5:14 AM Paul Ruizendaal wrote: > > On 29 Mar 2020, at 16:04, arnold at skeeve.com wrote: > > > > Paul Ruizendaal wrote: > > > >> Related is the question when the "file system switch" was added. It must > >> have been later than 1981 and before 1985, but I have not been able to > >> pinpoint it further. > > > > IIRC there was a "paper" (only an abstract) on the file system > > switch published in a USENIX conference proceedings. That woud help > > trace it down. > > I have that paper (“The Unix 8th Edition Network File System”), it was presented at a March 1985 ACM conference. However, there are indications that the roots of the file system switch existed earlier, possibly much earlier. > > I think Doug McIlroy once described 1973 as a pivotal year for Unix, with many concepts devised that would blossom in the following 3-5 years. I’m increasingly tempted to think that Summer ’81 - Summer ’82 was a similarly pivotal year. > > > Peter Weinberger, who did it, is at Google; you could ask him > > directly, as well. > > That is a good idea. If someone has the email address I’d appreciate an off list message. > > Paul > > > From pnr at planet.nl Mon Mar 30 16:25:18 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Mon, 30 Mar 2020 08:25:18 +0200 Subject: [TUHS] 8th Edition timeline In-Reply-To: <202003291404.02TE4dAI022916@freefriends.org> References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> Message-ID: >> Peter Weinberger, who did it, is at Google; you could ask him >> directly, as well. > > That is a good idea. If someone has the email address I’d appreciate an off list message. > Email address received off list - many thanks all. Paul From pnr at planet.nl Mon Mar 30 17:43:06 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Mon, 30 Mar 2020 09:43:06 +0200 Subject: [TUHS] 8th Edition timeline In-Reply-To: References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> Message-ID: > On 29 Mar 2020, at 23:48, Rob Pike wrote: > > I have a dog in the fight, having joined in June 1980, but that is not > a coincidence. The period of 1980-1982 was a big one for 127 (soon > 1127) as they were finally given the chance to grow, and I was one of > the lucky early hires in that burst. New blood brought in new ideas > and things happened fast. I had not realised that the Research group expanded in 1980, but it fits. > It was also the time of the VAX; the center's 11/780 arrived in late > 1980 I think, maybe early 1981. I did realise that bit, and it made me wonder if the ’73 burst was in part driven by the arrival of a 11/45. > Our first experiments with graphical terminals spanned 1980 to early 1981, Yes, as you may remember from this list I dove into that last November - schematics, tools & firmware. > using Greg Chesson's mux, Chesson’s MPX files remain a puzzle piece that is somewhat difficult to fit in the overall story, having so many aspects. It sits between Rand ports and SysIII fifo’s, experiments with non-blocking I/O, has aspects of pseudo-terminals, etc. I have not been able to figure out what immediate need they served, unless it was used in the first generation Datakit software (as MPX precedes the Jerq, that cannot have been the immediate need.) > but by late 1981 we were using Dennis's streams (only STREAMS when > they went to USG) and the select system call, which was by then running > in a merged Berkeley/Research Unix that eventually became the Eighth > Edition. To be honest, late 1981 sounds a bit too early for the merge. The 4.1 code was ready in June 1981 and the ’select’ system call was first proposed in July 1981, so it is possible. However, in the BSD line ’select’ was not fully implemented until March/April 1982. It is certainly possible, even likely, that ‘streams’ date from 1981 or earlier. Networks don’t mesh well with TTY line disciplines and clist buffering - that pain will have become apparent already in 1979. Maybe it was among the first things to be fixed when the VAX arrived. > My notebooks can probably lock down a lot of this as I was a prolific > note-taker back then, when they still made paper. If someday you have time for this, it would be much appreciated! Paul From robpike at gmail.com Mon Mar 30 19:06:24 2020 From: robpike at gmail.com (Rob Pike) Date: Mon, 30 Mar 2020 20:06:24 +1100 Subject: [TUHS] 8th Edition timeline In-Reply-To: References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> Message-ID: I've looked through my notes and unfortunately there's very little about this as the notes are mostly about graphics and physics. You're right I was a little early above, but the "merge" was not an event. It was a process that went on for months, years even. Dennis wrote the select we used, I'm almost certain of that. We (he and I) talked about its design at the time, after a trip he made to Berkeley about the DARPA Stuff. He had had many discussions with the BSD work going on and wanted to couple it to the streams work to make my graphical multiplexer work. Mpx was killing me (and killing the kernel), but I'm nearly certain that I was using select (with our form of pseudo-ttys) before or by very early 1982. I'm all but certain; we gave up on mpx(2) very early. Streams came later; Dennis's Show and Tell about them was early in 1982. I'm not sure, but we may have booted London & Reiser's VAX port first, but we moved to Berkeley code pretty early. We (Dennis mostly, but others) were talking to Berkeley often. We didn't just wait for a release; we were a part (not sure I could say now how important, but a part) of it from early on. -rob On Mon, Mar 30, 2020 at 6:44 PM Paul Ruizendaal wrote: > > > > On 29 Mar 2020, at 23:48, Rob Pike wrote: > > > > I have a dog in the fight, having joined in June 1980, but that is not > > a coincidence. The period of 1980-1982 was a big one for 127 (soon > > 1127) as they were finally given the chance to grow, and I was one of > > the lucky early hires in that burst. New blood brought in new ideas > > and things happened fast. > > I had not realised that the Research group expanded in 1980, but it fits. > > > It was also the time of the VAX; the center's 11/780 arrived in late > > 1980 I think, maybe early 1981. > > I did realise that bit, and it made me wonder if the ’73 burst was in part driven by the arrival of a 11/45. > > > Our first experiments with graphical terminals spanned 1980 to early 1981, > > Yes, as you may remember from this list I dove into that last November - schematics, tools & firmware. > > > using Greg Chesson's mux, > > Chesson’s MPX files remain a puzzle piece that is somewhat difficult to fit in the overall story, having so many aspects. It sits between Rand ports and SysIII fifo’s, experiments with non-blocking I/O, has aspects of pseudo-terminals, etc. I have not been able to figure out what immediate need they served, unless it was used in the first generation Datakit software (as MPX precedes the Jerq, that cannot have been the immediate need.) > > > but by late 1981 we were using Dennis's streams (only STREAMS when > > they went to USG) and the select system call, which was by then running > > in a merged Berkeley/Research Unix that eventually became the Eighth > > Edition. > > To be honest, late 1981 sounds a bit too early for the merge. The 4.1 code was ready in June 1981 and the ’select’ system call was first proposed in July 1981, so it is possible. However, in the BSD line ’select’ was not fully implemented until March/April 1982. > > It is certainly possible, even likely, that ‘streams’ date from 1981 or earlier. Networks don’t mesh well with TTY line disciplines and clist buffering - that pain will have become apparent already in 1979. Maybe it was among the first things to be fixed when the VAX arrived. > > > My notebooks can probably lock down a lot of this as I was a prolific > > note-taker back then, when they still made paper. > > If someday you have time for this, it would be much appreciated! > > Paul > From clemc at ccc.com Mon Mar 30 23:26:35 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 30 Mar 2020 09:26:35 -0400 Subject: [TUHS] 8th Edition timeline In-Reply-To: References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> Message-ID: Paul - see below.. On Mon, Mar 30, 2020 at 3:43 AM Paul Ruizendaal wrote: > To be honest, late 1981 sounds a bit too early for the merge. The 4.1 code > was ready in June 1981 and the ’select’ system call was first proposed in > July 1981, so it is possible. However, in the BSD line ’select’ was not > fully implemented until March/April 1982. > Please be careful here. 4.1BSD is different from the pre-'4.2' released and 4.2BSD itself. 4.1aBSD was the first pseudo release[1] that started to have the major surgery to support Bill's sockets idea and splice in the UCB rewrite of the BBN code. 4.1BSD was the first system for the Vax that really wide distribution. 'Anyone' with an AT&T license could get it and most people did. Remember this is the system that BBN (Gurwitz) did the original IP/TCP support (sans sockets - i.e. /dev/ip /dev/tcp ...). Your date of June '81 for the 4.1BSD release seems late, but I'll accept it. 3BSD was 1979, and I thought 4BSD was a year later, with 4.1BSD a few months after 4BSD (few people actually got 4BSD) That said, Bill and Sam did the heavy lifting on select(2) first in 4.1aBSD and there were some issues (again I have forgotten the details -- I do remember, I was working on my thesis and I had a do a huge rewrite of the AP kernel support to handle select(2) properly). I remember talking to Sam (arguing with him most probably) about it one night before it was fully created. I want to say, he had worked on something similar at the firm he was at (the firm name I now forget -- si-mumble -- they were in Mt. View) before he joined CRSG. I don't remember now the issues I had, but I do remember it was a bit of mess to support the way the AP hardware assumed it could do DMA on the UBA[2] [1] 4.1a/4.1b/4.1cBSD was officially internal to UCB and some ARPA-sites, although I would have expected someone like Dennis in 1127 to have been sent it also, as wnj was in the process of leaving for Sun and he took them with him. For instance, I would take 4.1c to Masscomp. The key is that these were not as widely distributed as 4.1BSD. 4.2BSD would really accelerate BSD UNIX uptake, because of the networking support but there was more than 2-3 years between 4.1BSD and 4.2BSD. [2] The AP's MMU/DMA interface at the time, was causing me great hair, and that was likely to have been part of the reason I wanted some help/changes in the KPI interface - which is actually funny, they eventually came with the CMU Mach MMU changes of 4.4BSD (which was much more friendly to a multiprocessor/coprocessor architecture). FWIW: I never got them when I was at UCB. -------------- next part -------------- An HTML attachment was scrubbed... URL: From reed at reedmedia.net Mon Mar 30 23:38:38 2020 From: reed at reedmedia.net (reed at reedmedia.net) Date: Mon, 30 Mar 2020 08:38:38 -0500 (CDT) Subject: [TUHS] 8th Edition timeline In-Reply-To: References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> Message-ID: On Mon, 30 Mar 2020, Clem Cole wrote: ... > kernel support to?handle?select(2) properly).? I remember talking to Sam (arguing > with him most probably) about it one night before it was fully created.? ?Iwant > to say, he had worked on something similar at the firm he was at (the firm > name I now forget -- si-mumble -- they were in Mt. View) before he joined > CRSG.? I don't remember now the issues I had, but I do remember it was a bit > of mess to support the way the AP hardware assumed it could do DMA on the > UBA[2] >From my notes: After graduating, Leffler got a job in the Silicon Valley at Sytek, a small firm in Sunnyvale. % CITE: archives/csrg-archives/disk1/mnt/2.79/usenet/netdir/sites/sytek They used a PDP 11/45 and moved to a VAX 11/750 running VM/UNIX with four 300 baud ACU's (Automatic Call Units aka ``modems``) --- they marketed a broadband network to connect thousands of stations using coaxial cable over a distance of 40 miles. % maybe CITE: http://books.google.com/books?id=AyZfLaxYr9YC&pg=PA43&lpg=PA43&dq= %28408%29+734-9000++sytek&source=bl&ots=SYGI9rBAOV&sig=rxBo1SgfqdTLuNotXaa59nrR6 C4&hl=en&sa=X&ei=CzbwUJTjI6HN2QX48oGIAw&ved=0CEAQ6AEwBA#v=onepage&q=%28408%29%20 734-9000%20%20sytek&f=false Since they were using a VAX and he had kept in touch with Joy, he was a beta-test site for 4BSD.\cite{salus1994} % NOTE: about sytek: an ad to find employees %http://books.google.com/books?id=ChMAmfS1nEkC&pg=PA85&lpg=PA85&dq=%28408%29+734 -9000++sytek&source=bl&ots=aS_yQznXAy&sig=6vA7NTbspu3gcf_jT2sOiUSRPZU&hl=en&sa=X &ei=CzbwUJTjI6HN2QX48oGIAw&ved=0CC4Q6AEwAA#v=onepage&q=%28408%29%20734-9000%20%2 0sytek&f=false % NOTE: not to take over since no Sun then Leffler went to Berkeley to work with Joy\cite{billshannon2} in the fall of 1981.\cite{salus1994}. From clemc at ccc.com Tue Mar 31 00:41:22 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 30 Mar 2020 10:41:22 -0400 Subject: [TUHS] 8th Edition timeline In-Reply-To: References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> Message-ID: Jeremy -- Sytek - that was it -- thank you. Sunnyvale/Mt. View -- who can tell the difference? ;-) Thanks again, Clem On Mon, Mar 30, 2020 at 10:21 AM wrote: > On Mon, 30 Mar 2020, Clem Cole wrote: > > ... > > kernel support to?handle?select(2) properly).? I remember talking to Sam > (arguing > > with him most probably) about it one night before it was fully created.? > ?Iwant > > to say, he had worked on something similar at the firm he was at (the > firm > > name I now forget -- si-mumble -- they were in Mt. View) before he joined > > CRSG.? I don't remember now the issues I had, but I do remember it was a > bit > > of mess to support the way the AP hardware assumed it could do DMA on the > > UBA[2] > > From my notes: > > After graduating, Leffler got a job in the Silicon Valley > at Sytek, a small firm in Sunnyvale. > % CITE: archives/csrg-archives/disk1/mnt/2.79/usenet/netdir/sites/sytek > They used a PDP 11/45 and moved to a VAX 11/750 running VM/UNIX > with four 300 baud ACU's (Automatic Call Units aka ``modems``) --- > they marketed a broadband network to > connect thousands of stations using coaxial cable over a distance > of 40 miles. > % maybe CITE: > http://books.google.com/books?id=AyZfLaxYr9YC&pg=PA43&lpg=PA43&dq= > > %28408%29+734-9000++sytek&source=bl&ots=SYGI9rBAOV&sig=rxBo1SgfqdTLuNotXaa59nrR6 > > C4&hl=en&sa=X&ei=CzbwUJTjI6HN2QX48oGIAw&ved=0CEAQ6AEwBA#v=onepage&q=%28408%29%20 > 734-9000%20%20sytek&f=false > > Since they were using a VAX and he had kept in touch with Joy, he was > a beta-test site for 4BSD.\cite{salus1994} > > % NOTE: about sytek: an ad to find employees > % > http://books.google.com/books?id=ChMAmfS1nEkC&pg=PA85&lpg=PA85&dq=%28408%29+734 > > -9000++sytek&source=bl&ots=aS_yQznXAy&sig=6vA7NTbspu3gcf_jT2sOiUSRPZU&hl=en&sa=X > > &ei=CzbwUJTjI6HN2QX48oGIAw&ved=0CC4Q6AEwAA#v=onepage&q=%28408%29%20734-9000%20%2 > 0sytek&f=false > > > % NOTE: not to take over since no Sun then > Leffler went to Berkeley to work with Joy\cite{billshannon2} in the > fall of 1981.\cite{salus1994}. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nobozo at gmail.com Tue Mar 31 01:06:54 2020 From: nobozo at gmail.com (Jon Forrest) Date: Mon, 30 Mar 2020 08:06:54 -0700 Subject: [TUHS] Sytek (was Re: 8th Edition timeline) In-Reply-To: References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> Message-ID: <6a9a9180-e247-2072-369e-db93332a0b4e@gmail.com> On 3/30/2020 7:41 AM, Clem Cole wrote: > Jeremy -- Sytek - that was it -- thank you.   Sunnyvale/Mt. View -- who > can tell the difference?  ;-) Here's a story about Sytek (I hope I haven't told it before - I'm starting to forget who I've told stories to). In 1977/1978 I was an extremely junior engineeer at Ford Aerospace. I was in the group that was both acting as the main Unix promoter in Ford, and also was making a provably-secure version of Unix called KSOS. The group had a bunch of excellent senior people (e.g. John Nagle, Tom Berson, Ken Biba, Mike Pliner, Mike Padlipsky). This was my first real job in the software industry so I didn't know what to expect. After about 6 months I noticed that there were a lot of closed-door meetings going on. I thought this was normal. One day I was walking by an office when the door opened up and I was invited in. I didn't know if I had done something wrong and was about to be fired, or what. I turned out that I had nothing to worry about. What was happening was that some of the people in the group were going to leave Ford to start their own company. They needed some junior people also, and they asked me to join them. I gave it some thought but turned them down because I didn't want to live in Silicon Valley anymore, and wanted to return to Santa Barabara where I belonged. The company that they started was Sytek, which got the contract from IBM to create NetBIOS. They also did other networking work that has been mentioned in this thread. If I had gone to Sytex I would have been employe #8 with lots of stock options. I often wonder how my life would have been different if I had done this. I suspect many people on TUHS have similar stories about the ones that got away. In fact, this might be a good topic for a new thread. Cordially, Jon Forrest From pnr at planet.nl Tue Mar 31 02:02:11 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Mon, 30 Mar 2020 18:02:11 +0200 Subject: [TUHS] 8th Edition timeline In-Reply-To: References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> Message-ID: <6AE2011D-32EF-4E1E-A231-9DFAC683FE48@planet.nl> Comments in line: > On 30 Mar 2020, at 15:26, Clem Cole wrote: > > Paul - see below.. > > On Mon, Mar 30, 2020 at 3:43 AM Paul Ruizendaal wrote: > To be honest, late 1981 sounds a bit too early for the merge. The 4.1 code was ready in June 1981 and the ’select’ system call was first proposed in July 1981, so it is possible. However, in the BSD line ’select’ was not fully implemented until March/April 1982. > Please be careful here. 4.1BSD is different from the pre-'4.2' released and 4.2BSD itself. 4.1aBSD was the first pseudo release[1] that started to have the major surgery to support Bill's sockets idea and splice in the UCB rewrite of the BBN code. 4.1BSD was the first system for the Vax that really wide distribution. 'Anyone' with an AT&T license could get it and most people did. Remember this is the system that BBN (Gurwitz) did the original IP/TCP support (sans sockets - i.e. /dev/ip /dev/tcp ...). Your date of June '81 for the 4.1BSD release seems late, but I'll accept it. 3BSD was 1979, and I thought 4BSD was a year later, with 4.1BSD a few months after 4BSD (few people actually got 4BSD) I am aware of that. “20 years of Berkely Unix” says November 1980 for 4BSD and June 1981 for 4.1BSD. From the SCCS log I’d say November 9th and July 1st respectively. As far as I can tell CSRG integrated the BBN stack with (almost) 4.1 in April 1981 - most of that is in a separate tree, with a few #ifdef’s in the main tree. From SCCS it seems that implementation work on sockets started in Oct/Nov 1981 and 4.1a was released in April or March 1982. From the partial coverage in SCCS I’d say there is no BBN code in 4.1a anymore. The socket API in that release was different from what ended being in 4.1c and 4.2, and seems to have had some wider visibility in 2.9BSD and UniPlus System V - both of which used the 4.1a API (and probably code base). > That said, Bill and Sam did the heavy lifting on select(2) first in 4.1aBSD and there were some issues (again I have forgotten the details -- I do remember, I was working on my thesis and I had a do a huge rewrite of the AP kernel support to handle select(2) properly). I remember talking to Sam (arguing with him most probably) about it one night before it was fully created. I want to say, he had worked on something similar at the firm he was at (the firm name I now forget -- si-mumble -- they were in Mt. View) before he joined CRSG. I don't remember now the issues I had, but I do remember it was a bit of mess to support the way the AP hardware assumed it could do DMA on the UBA[2] > [1] 4.1a/4.1b/4.1cBSD was officially internal to UCB and some ARPA-sites, although I would have expected someone like Dennis in 1127 to have been sent it also, as wnj was in the process of leaving for Sun and he took them with him. For instance, I would take 4.1c to Masscomp. The key is that these were not as widely distributed as 4.1BSD. 4.2BSD would really accelerate BSD UNIX uptake, because of the networking support but there was more than 2-3 years between 4.1BSD and 4.2BSD. Yes, Summer 81 - Summer 82 must have been a vibrant year, with much interaction between various Unix groups. By the way, dmr was on the 4.2BSD steering group. > [2] The AP's MMU/DMA interface at the time, was causing me great hair, and that was likely to have been part of the reason I wanted some help/changes in the KPI interface - which is actually funny, they eventually came with the CMU Mach MMU changes of 4.4BSD (which was much more friendly to a multiprocessor/coprocessor architecture). FWIW: I never got them when I was at UCB. From clemc at ccc.com Tue Mar 31 05:31:13 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 30 Mar 2020 15:31:13 -0400 Subject: [TUHS] 8th Edition timeline In-Reply-To: <6AE2011D-32EF-4E1E-A231-9DFAC683FE48@planet.nl> References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> <6AE2011D-32EF-4E1E-A231-9DFAC683FE48@planet.nl> Message-ID: On Mon, Mar 30, 2020 at 12:02 PM Paul Ruizendaal wrote: > Comments in line: > > > On 30 Mar 2020, at 15:26, Clem Cole wrote: > Your date of June '81 for the 4.1BSD release seems late, but I'll accept > it. 3BSD was 1979, and I thought 4BSD was a year later, with 4.1BSD a few > months after 4BSD (few people actually got 4BSD) > > I am aware of that. “20 years of Berkely Unix” says November 1980 for 4BSD > and June 1981 for 4.1BSD. From the SCCS log I’d say November 9th and July > 1st respectively. Ok, that sounds right 6-7 months between them. That was the time of the 'FASTVAX' work and the fight with Stanford over if BSD or VMS was to be the official Arpa OS. I remember Joy was incredibly prolific during that time. From a user standpoint, 4BSD and 4.1BSD are really similar, but he was hacking the kernel. It seemed like a new change came out all the time. The CAD group (where I was) always wanted the fastest system, so there was pressure on me to follow him, but I was trying to make progress on the AP work (my thesis) so I was a little loath to take the churn. > > > As far as I can tell CSRG integrated the BBN stack with (almost) 4.1 in > April 1981 - most of that is in a separate tree, with a few #ifdef’s in the > main tree. I'm pretty sure that was mostly Eric Cooper (ecc) IIRC. We had a tape directly from Gurwitz running in the CAD lab in Cory Hall, running on Xerox 3M Ethernet and because we needed it between our three 780s systems. We had ordered the 3C501's from 3COM and we were going to be the Beta for the Interlan board - which I wrote the original driver and gave it to Sam. I'm not sure when IngVAX arrived, Eric Schmidt had Ing70 downstairs (and that was the ArpaNet connect via a VDH up the hill to the IMP at LBL. We had been running BerkNET over serial lines. Ernie and Kim were the two 780s over in Evan's (the big pile of 750s and the C30 IMP were still in the future). > From SCCS it seems that implementation work on sockets started in Oct/Nov > 1981 and 4.1a was released in April or March 1982. That sounds about right. It was a bunch of wnj marathon hacking sessions. > From the partial coverage in SCCS I’d say there is no BBN code in 4.1a > anymore. Mumble ... I sort of have a hard time with that. I knew and worked with both of them in my day and have heard both sides of the story. Joy had Gurwitz's code and was hacking it - that is just a fact. Claiming pure authorship is a tad extreme. For instance, Andy VanDam told me that the whole mbuf's stuff is something Rob originally wrote for a project for him at Brown when he was an undergrad. Rob brought it BBN and he used it for the OS-independent TCP (i.e. the HP3000 version) originally and it landed in the UNIX version. Joy hacked on it, but that's a data structure directly from Rob. On the other hand, Joy types open curly brace, close curly brace, and hacks in code the fastest of anyone I ever knew (someone, maybe Mary Ann Horton, once said he 'wrote code at 9600'). But if you look at it, its sometimes not pretty. It works and its usually fast, which is hard to argue with. > The socket API in that release was different from what ended being in 4.1c > and 4.2, Definitely. I thought other hands besides Joy started to mess with it (like Sam) > and seems to have had some wider visibility in 2.9BSD and UniPlus System V > - both of which used the 4.1a API (and probably code base). Yeah, Keith was taking things was Sam at pretty hectic pace trying to keep the 11's running. I would guess that took more of a 'good enough' attitude, they got the TCP stack in the limited address space and did not want to try much more > > > Yes, Summer 81 - Summer 82 must have been a vibrant year, with much > interaction between various Unix groups. It was a fun time to be apart of it all. > By the way, dmr was on the 4.2BSD steering group. Yes, I know. That's how I first got to know him. -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Tue Mar 31 09:23:27 2020 From: crossd at gmail.com (Dan Cross) Date: Mon, 30 Mar 2020 19:23:27 -0400 Subject: [TUHS] Vint Cerf tests positive for COVID-19. Message-ID: He announced it on Twitter. I'm sure we all wish him well and a speedy recovery. Note, his tweet says that he is recovering. https://twitter.com/vgcerf/status/1244636584508604417?s=20 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pnr at planet.nl Tue Mar 31 12:44:06 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 31 Mar 2020 04:44:06 +0200 Subject: [TUHS] 8th Edition timeline In-Reply-To: References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> <6AE2011D-32EF-4E1E-A231-9DFAC683FE48@planet.nl> Message-ID: <67CAA7B3-798E-4F19-A06A-E41D95D22684@planet.nl> > [...] the whole mbuf's stuff is something [...] I would prefer not to go into the BSD code history at this point in time, as I’d like to stay focused on what was going on at Bell Labs between 7th and 8th edition, as far as networking is concerned. Of course it was all interconnecting, and the topic of mbuf's is a nice arc to take us ultimately back too streams. Looking at network buffering in Unix, I think the story begins in 1973 with Spider. In Spider, the network driver was a character mode device, but used disk block buffering for storage. This worked well, because the Spider interface sent/received data through DMA, in cells of 256 bytes and at speeds comparable to contemporary disks: https://minnie.tuhs.org/cgi-bin/utree.pl?file=V4/nsys/dmr/tdir/tiu.c. In 1975 the UoI guys doing Arpa Unix had to interface to an IMP, which was much slower than a disk but much faster than a terminal. They concluded that neither clists not disk buffers were a good match and Steve Bunch came up with ’netbuf’s, which were 64 byte blocks (carved out of a pool of disk blocks). Later (1978?) this code was modified to optionally use separate buffer heads and buffer data, as part of the UoC buffer mod’s (to alleviate the pressure on a limited address space): https://minnie.tuhs.org/cgi-bin/utree.pl?file=SRI-NOSC/h/net/netbuf.h In 1979 the first Datakit software was done (Chesson), where I don’t have much information. My hypothesis is that this used clists, but I’m still trying to find a copy of his paper about this implementation. Maybe the driver used a precursor to a streams like data structure. Maybe this is also where MPX files fit in. Unfortunately, currently I just don’t know. In 1980 new TCP software for the VAX is being written, and again it needs to find a data buffer approach. This is ‘mbuf’s, which are similar to the earlier ’netbuf’s. The main changes are that the size is increased to 128 bytes and that a start offset is added, which makes it possible to remove headers without copying the buffer. The ‘memberof’ field is replaced by a second linkage pointer. The UoC idea to separate the header and data is dropped again. They (mbufs) may have other origins as well, but as far as Unix networking is concerned they are an evolution of the Arpa Unix ‘netbuf’s: https://minnie.tuhs.org/cgi-bin/utree.pl?file=BBN-Vax-TCP/bbnnet/mbuf.h Then we get to (late) 1981. On the Labs side streams are created, in my current understanding mainly to support networking and in particular Datakit. The difference with Spider is that the Datakit interface was not DMA based and that the data arrives in 9 bit envelopes instead of 8 bit bytes. The 9th bit distinguishes between data bytes and control bytes. Again neither clists nor disk buffers are a good match for buffering and a new solution is created. Looking at buffers, these consist of a header and a separate datablock. The datablock can be of various sizes: 4, 16, 64 and 1024 bytes - a match with the varying nature of network traffic. Each block has a type (a.o. DATA and CONTROL) which has various uses, but certainly gives a home to the 9th Datakit bit where needed: https://github.com/Alhadis/Research-Unix-v8/blob/master/v8/usr/include/sys/stream.h https://github.com/Alhadis/Research-Unix-v8/blob/master/v8/usr/sys/dev/stream.c Over on the west coast the mbuf structure is extended with a new field, “type” at the same time, and it transitions from being a pure network data buffer to a more generic buffer. It remains at a fixed size of 128 bytes with inline data; the option to store data in a separate block of varying size will only come back late in 4.3BSD. For the 4.2 version: https://minnie.tuhs.org/cgi-bin/utree.pl?file=4.2BSD/usr/src/sys/h/mbuf.h When viewed at this code level it would seem to me that by various routes everybody had arrived at more or less the same the point in network buffering approach by 1981. That has some interesting implications. At this level, is there really a difference between a pipe implemented as cross connected streams and a pipe implemented as cross connected sockets? Is an unbound socket not about the same as an unconnected stream head? But before getting into such discussions, I’d really like to get the development story from 7th to 8th edition as clear and precise as possible after 40 years. > On 30 Mar 2020, at 21:31, Clem Cole wrote: > > > > On Mon, Mar 30, 2020 at 12:02 PM Paul Ruizendaal wrote: > Comments in line: > > > On 30 Mar 2020, at 15:26, Clem Cole wrote: > Your date of June '81 for the 4.1BSD release seems late, but I'll accept it. 3BSD was 1979, and I thought 4BSD was a year later, with 4.1BSD a few months after 4BSD (few people actually got 4BSD) > > I am aware of that. “20 years of Berkely Unix” says November 1980 for 4BSD and June 1981 for 4.1BSD. From the SCCS log I’d say November 9th and July 1st respectively. > Ok, that sounds right 6-7 months between them. That was the time of the 'FASTVAX' work and the fight with Stanford over if BSD or VMS was to be the official Arpa OS. I remember Joy was incredibly prolific during that time. From a user standpoint, 4BSD and 4.1BSD are really similar, but he was hacking the kernel. It seemed like a new change came out all the time. The CAD group (where I was) always wanted the fastest system, so there was pressure on me to follow him, but I was trying to make progress on the AP work (my thesis) so I was a little loath to take the churn. > > > > > As far as I can tell CSRG integrated the BBN stack with (almost) 4.1 in April 1981 - most of that is in a separate tree, with a few #ifdef’s in the main tree. > I'm pretty sure that was mostly Eric Cooper (ecc) IIRC. We had a tape directly from Gurwitz running in the CAD lab in Cory Hall, running on Xerox 3M Ethernet and because we needed it between our three 780s systems. We had ordered the 3C501's from 3COM and we were going to be the Beta for the Interlan board - which I wrote the original driver and gave it to Sam. I'm not sure when IngVAX arrived, Eric Schmidt had Ing70 downstairs (and that was the ArpaNet connect via a VDH up the hill to the IMP at LBL. We had been running BerkNET over serial lines. Ernie and Kim were the two 780s over in Evan's (the big pile of 750s and the C30 IMP were still in the future). > > > > From SCCS it seems that implementation work on sockets started in Oct/Nov 1981 and 4.1a was released in April or March 1982. > That sounds about right. It was a bunch of wnj marathon hacking sessions. > > > From the partial coverage in SCCS I’d say there is no BBN code in 4.1a anymore. > Mumble ... I sort of have a hard time with that. I knew and worked with both of them in my day and have heard both sides of the story. Joy had Gurwitz's code and was hacking it - that is just a fact. Claiming pure authorship is a tad extreme. For instance, Andy VanDam told me that the whole mbuf's stuff is something Rob originally wrote for a project for him at Brown when he was an undergrad. Rob brought it BBN and he used it for the OS-independent TCP (i.e. the HP3000 version) originally and it landed in the UNIX version. Joy hacked on it, but that's a data structure directly from Rob. On the other hand, Joy types open curly brace, close curly brace, and hacks in code the fastest of anyone I ever knew (someone, maybe Mary Ann Horton, once said he 'wrote code at 9600'). But if you look at it, its sometimes not pretty. It works and its usually fast, which is hard to argue with. > > > > The socket API in that release was different from what ended being in 4.1c and 4.2, > Definitely. I thought other hands besides Joy started to mess with it (like Sam) > > > and seems to have had some wider visibility in 2.9BSD and UniPlus System V - both of which used the 4.1a API (and probably code base). > Yeah, Keith was taking things was Sam at pretty hectic pace trying to keep the 11's running. I would guess that took more of a 'good enough' attitude, they got the TCP stack in the limited address space and did not want to try much more > > > Yes, Summer 81 - Summer 82 must have been a vibrant year, with much interaction between various Unix groups. > It was a fun time to be apart of it all. > > > By the way, dmr was on the 4.2BSD steering group. > Yes, I know. That's how I first got to know him. From dave at horsfall.org Tue Mar 31 13:13:45 2020 From: dave at horsfall.org (Dave Horsfall) Date: Tue, 31 Mar 2020 14:13:45 +1100 (EST) Subject: [TUHS] Vint Cerf tests positive for COVID-19. In-Reply-To: References: Message-ID: On Mon, 30 Mar 2020, Dan Cross wrote: > He announced it on Twitter. I'm sure we all wish him well and a speedy > recovery. Note, his tweet says that he is recovering. Indeed. On a whim I did a search for him, and got: Get Vint Cerf With Fast and Free Shipping on eBay. Looking For Vint Cerf? We Have Almost Everything on eBay. I'm sure that he'll be happy to hear that... -- Dave From doug at cs.dartmouth.edu Tue Mar 31 13:40:16 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Mon, 30 Mar 2020 23:40:16 -0400 Subject: [TUHS] weird \s Message-ID: <202003310340.02V3eGM9076785@tahoe.cs.Dartmouth.EDU> Thanks, everyone, for pointing out the "accident of history". This is a little morality tale about clever programming. Joe Ossanna would never have tokenized \s constructs that way if the CAT typesetter had been capable of a wider range of type sizes. Brian seems to have kept it for backward compatibility. But groff was perfectly willing to break backward compatibility in tokenizing command names. Why not in \s? I encourage Branden to go whole hog and let \s123DEWEY WINS produce a banner headline. Doug