From fmouse-namazu at fmp.com Fri Jan 12 12:23:23 2007 From: fmouse-namazu at fmp.com (Lindsay Haisley) Date: Fri Jan 12 12:29:05 2007 Subject: [Namazu-users-en] Problems with date sorts Message-ID: <20070112032322.GA16647@fmp.com> I'm running into a rather nasty problem with date sorting on Mailman pipermail archives. When I sort on date:early or date:late there appears to be some other sort being applied, although if I do a date sort in the reverse order the order of the messages is indeed reversed, indicating that the sort is working, albeit with an incorrect algorithm. Sometimes some of the posts are close to being in chronological order, sometimes there are sections that are in reverse order of what they should be, and sometimes the order appears random. Overall the sort is failing, although the sort order is the same every time so the problem is reproducable and consistent. I'm not seeing any errors, either when I run mknmz or when I do a search, so I don't know whether there's incorrect information is in the indexes or if the correct information is there and is being mishandled. Has anyone else had problems similar to this? I'm trying to read code, but the parts are scattered here and there between perl perl programs and C code and I'm wasting a lot of time and don't seem to be any closer to a solution. Any help would be appreciated, or pointers to any diagnistics that I could run which might help. -- Lindsay Haisley | "Fighting against human | PGP public key FMP Computer Services | creativity is like | available at 512-259-1190 | trying to eradicate | http://www.fmp.com | dandelions" | | (Pamela Jones) | From yw3t-trns at asahi-net.or.jp Sat Jan 13 00:18:26 2007 From: yw3t-trns at asahi-net.or.jp (Tadamasa Teranishi) Date: Sat Jan 13 00:18:35 2007 Subject: [Namazu-users-en] Re: Problems with date sorts References: <20070112032322.GA16647@fmp.com> Message-ID: <45A7A6C2.86AD2F9E@asahi-net.or.jp> Lindsay Haisley wrote: > > I'm running into a rather nasty problem with date sorting on Mailman pipermail > archives. When I sort on date:early or date:late there appears to be some > other sort being applied, although if I do a date sort in the reverse order the > order of the messages is indeed reversed, indicating that the sort is working, > albeit with an incorrect algorithm. Does date information accurately follow the form of RFC2822 by all documents of MailMan? Is there mail with an illegal Date: field ? Please show the Date: field of the mail. -- ===================================================================== TADAMASA TERANISHI yw3t-trns@asahi-net.or.jp http://www.asahi-net.or.jp/~yw3t-trns/index.htm Key fingerprint = 474E 4D93 8E97 11F6 662D 8A42 17F5 52F4 10E7 D14E From fmouse-namazu at fmp.com Sat Jan 13 08:19:00 2007 From: fmouse-namazu at fmp.com (Lindsay Haisley) Date: Sat Jan 13 08:19:08 2007 Subject: [Namazu-users-en] Re: Problems with date sorts In-Reply-To: <45A7A6C2.86AD2F9E@asahi-net.or.jp> References: <20070112032322.GA16647@fmp.com> <45A7A6C2.86AD2F9E@asahi-net.or.jp> Message-ID: <20070112231900.GA11197@fmp.com> Thus spake Tadamasa Teranishi on Fri, Jan 12, 2007 at 09:18:26AM CST > Lindsay Haisley wrote: > > > > I'm running into a rather nasty problem with date sorting on Mailman pipermail > > archives. When I sort on date:early or date:late there appears to be some > > other sort being applied, although if I do a date sort in the reverse order the > > order of the messages is indeed reversed, indicating that the sort is working, > > albeit with an incorrect algorithm. > > Does date information accurately follow the form of RFC2822 by > all documents of MailMan? > > Is there mail with an illegal Date: field ? > > Please show the Date: field of the mail. OK, here is an example. I used the following query: http://www.kca-tx.org/mailman/kca/namazu.cgi?query=Laptop&submit=Search%21&idxname=kca&max=100&result=short&sort=date%3Aearly Here's the result: 1. win 98SE (score: 2) /pipermail/kca/2002-September/000192.html (4,152 bytes) 2. Linux install plus a note on jedit (score: 2) /pipermail/kca/2002-July/000052.html (4,432 bytes) 3. Canon BJC-2100, Restart in DOS mode (score: 2) /pipermail/kca/2002-August/000103.html (3,073 bytes) 4. March Newscard (score: 2) /pipermail/kca/2003-March/000353.html (4,296 bytes) 5. New TurboTax "feature" (score: 2) /pipermail/kca/2003-January/000331.html (6,865 bytes) You can see from path names that these are out of order. Here are the Date fields in each of these, copy-n-pasted from the files themselves: Sun Sep 22 14:07:51 CDT 2002 Fri Jul 19 11:54:00 CDT 2002 Fri Aug 23 08:44:57 CDT 2002 Mon Mar 3 10:14:12 CST 2003 Tue Jan 14 17:37:11 CST 2003 The date information isn't in a standard RFC2822 header format once the files are in a pipermail archive, but embedded in HTML markup, e.g.: New TurboTax "feature"

New TurboTax "feature"

Dale Cockle k5jic at kca-tx.org
Tue Jan 14 17:37:11 CST 2003 etc.... Could that be a problem? Should I perhaps be indexing the mbox file? Would namazu understand that better? -- Lindsay Haisley | "Fighting against human | PGP public key FMP Computer Services | creativity is like | available at 512-259-1190 | trying to eradicate | http://www.fmp.com | dandelions" | | (Pamela Jones) | From fmouse-namazu at fmp.com Sat Jan 13 08:48:15 2007 From: fmouse-namazu at fmp.com (Lindsay Haisley) Date: Sat Jan 13 08:48:18 2007 Subject: [Namazu-users-en] Further note on problems with date sorts Message-ID: <20070112234815.GC11197@fmp.com> I should note that my mknmz invocation is as follows: /usr/bin/mknmz -O $1 --media-type='text/html; x-type=pipermail' \ --config=/home/nmz/$1/.mknmzrc --template-dir=$1 \ /usr/local/mailman/archives/private/$1 $1 is the directory, named for the list name. -- Lindsay Haisley | "Fighting against human | PGP public key FMP Computer Services | creativity is like | available at 512-259-1190 | trying to eradicate | http://www.fmp.com | dandelions" | | (Pamela Jones) | From fmouse-namazu at fmp.com Sat Jan 13 11:27:43 2007 From: fmouse-namazu at fmp.com (Lindsay Haisley) Date: Sat Jan 13 11:27:46 2007 Subject: [Namazu-users-en] Storage of date information Message-ID: <20070113022743.GA13100@fmp.com> Some questions .... How is date information stored in the namazu index files? In which files is it stored, and if it's in a compressed (binary) form, how can this be viewed? Where is the association between pipermail html files and dates stored, or is this association determined at run time by namazu.cgi or the namazu CLI utility? In other words, if I want to look at exactly how namazu is determining the order for a sort by date, and debug the process, where do I need to look for the code? -- Lindsay Haisley | "Fighting against human | PGP public key FMP Computer Services | creativity is like | available at 512-259-1190 | trying to eradicate | http://www.fmp.com | dandelions" | | (Pamela Jones) | From fmouse-namazu at fmp.com Sat Jan 13 11:32:30 2007 From: fmouse-namazu at fmp.com (Lindsay Haisley) Date: Sat Jan 13 11:32:32 2007 Subject: [Namazu-users-en] Re: Further note on problems with date sorts Message-ID: <20070113023230.GB13100@fmp.com> I did a bit of work on this. Looks like pipermail.pl is correctly grabbing the dates from the pipermail files. The problem is elsewhere. Thus spake Lindsay Haisley on Fri, Jan 12, 2007 at 05:48:15PM CST > I should note that my mknmz invocation is as follows: > > /usr/bin/mknmz -O $1 --media-type='text/html; x-type=pipermail' \ > --config=/home/nmz/$1/.mknmzrc --template-dir=$1 \ > /usr/local/mailman/archives/private/$1 > > $1 is the directory, named for the list name. > > -- > Lindsay Haisley | "Fighting against human | PGP public key > FMP Computer Services | creativity is like | available at > 512-259-1190 | trying to eradicate | > http://www.fmp.com | dandelions" | > | (Pamela Jones) | > _______________________________________________ > Namazu-users-en mailing list > Namazu-users-en@namazu.org > http://www.namazu.org/cgi-bin/mailman/listinfo/namazu-users-en -- Lindsay Haisley | "Fighting against human | PGP public key FMP Computer Services | creativity is like | available at 512-259-1190 | trying to eradicate | http://www.fmp.com | dandelions" | | (Pamela Jones) | From yw3t-trns at asahi-net.or.jp Sat Jan 13 13:58:00 2007 From: yw3t-trns at asahi-net.or.jp (Tadamasa Teranishi) Date: Sat Jan 13 13:58:11 2007 Subject: [Namazu-users-en] Re: Problems with date sorts References: <20070112032322.GA16647@fmp.com> <45A7A6C2.86AD2F9E@asahi-net.or.jp> <20070112231900.GA11197@fmp.com> Message-ID: <45A866D8.BC7A15BB@asahi-net.or.jp> Lindsay Haisley wrote: > > > Does date information accurately follow the form of RFC2822 by > > all documents of MailMan? > > > > Is there mail with an illegal Date: field ? > > > > Please show the Date: field of the mail. > > OK, here is an example. I used the following query: > > http://www.kca-tx.org/mailman/kca/namazu.cgi?query=Laptop&submit=Search%21&idxname=kca&max=100&result=short&sort=date%3Aearly > > Here's the result: > > 1. win 98SE (score: 2) > /pipermail/kca/2002-September/000192.html (4,152 bytes) ... > You can see from path names that these are out of order. Here are the Date > fields in each of these, copy-n-pasted from the files themselves: > > Sun Sep 22 14:07:51 CDT 2002 > Fri Jul 19 11:54:00 CDT 2002 > Fri Aug 23 08:44:57 CDT 2002 > Mon Mar 3 10:14:12 CST 2003 > Tue Jan 14 17:37:11 CST 2003 To begin with, Date of pipermail was not RFC2822 form. However, Date of pipermail is correctly reflected in the field 'Date'. Then, let's examine the time stamp of pipermail file next. For instance, please confirm the following file and confirm the date by 'ls -alF'. /pipermail/kca/2002-September/000192.html /pipermail/kca/2002-July/000052.html /pipermail/kca/2002-August/000103.html /pipermail/kca/2003-March/000353.html /pipermail/kca/2003-January/000331.html It might be not corresponding to Date of the content of the time stamp and the file. The date sorting is a function to sort the time order of the stamp. There are two kinds of methods of solving this problem. One is a method of changing the time stamp of the file according to information on Date of contents of the file. Another one is a method of using the field sorting UTC function. Do you know the method of setting the field sorting UTC function? -- ===================================================================== TADAMASA TERANISHI yw3t-trns@asahi-net.or.jp http://www.asahi-net.or.jp/~yw3t-trns/index.htm Key fingerprint = 474E 4D93 8E97 11F6 662D 8A42 17F5 52F4 10E7 D14E From yw3t-trns at asahi-net.or.jp Sat Jan 13 14:07:11 2007 From: yw3t-trns at asahi-net.or.jp (Tadamasa Teranishi) Date: Sat Jan 13 14:07:20 2007 Subject: [Namazu-users-en] Re: Storage of date information References: <20070113022743.GA13100@fmp.com> Message-ID: <45A868FF.B000495F@asahi-net.or.jp> Lindsay Haisley wrote: > > How is date information stored in the namazu index files? In which files is it > stored, and if it's in a compressed (binary) form, how can this be viewed? > Where is the association between pipermail html files and dates stored, or is > this association determined at run time by namazu.cgi or the namazu CLI > utility? The format of the index file is being written here. http://www.namazu.org/doc/nmz.html.en The time stamp is preserved in NMZ.t. http://www.namazu.org/doc/nmz.html.en#t The date field is preserved in NMZ.field.date. http://www.namazu.org/doc/nmz.html.en#field -- ===================================================================== TADAMASA TERANISHI yw3t-trns@asahi-net.or.jp http://www.asahi-net.or.jp/~yw3t-trns/index.htm Key fingerprint = 474E 4D93 8E97 11F6 662D 8A42 17F5 52F4 10E7 D14E From yw3t-trns at asahi-net.or.jp Sat Jan 13 16:37:35 2007 From: yw3t-trns at asahi-net.or.jp (Tadamasa Teranishi) Date: Sat Jan 13 16:37:45 2007 Subject: [Namazu-users-en] Re: Problems with date sorts References: <20070112032322.GA16647@fmp.com> Message-ID: <45A88C3F.FEFC453C@asahi-net.or.jp> Please send the question addressed to ML though the mail on which it questioned an individual appropriating was gotten. If mail can be sent addressed to ML, I will answer. -- ===================================================================== TADAMASA TERANISHI yw3t-trns@asahi-net.or.jp http://www.asahi-net.or.jp/~yw3t-trns/index.htm Key fingerprint = 474E 4D93 8E97 11F6 662D 8A42 17F5 52F4 10E7 D14E From fmouse-namazu at fmp.com Sun Jan 14 03:14:51 2007 From: fmouse-namazu at fmp.com (Lindsay Haisley) Date: Sun Jan 14 03:14:57 2007 Subject: [Namazu-users-en] Re: Problems with date sorts In-Reply-To: <1168711030.5119.6.camel@localhost.localdomain> References: <1168711030.5119.6.camel@localhost.localdomain> Message-ID: <20070113181451.GA12872@fmp.com> On Sat, 2007-01-13 at 13:58 +0900, Tadamasa Teranishi wrote: > Lindsay Haisley wrote: > > > > > Does date information accurately follow the form of RFC2822 by > > > all documents of MailMan? > > > > > > Is there mail with an illegal Date: field ? > > > > > > Please show the Date: field of the mail. > > > > OK, here is an example. I used the following query: > > > > http://www.kca-tx.org/mailman/kca/namazu.cgi?query=Laptop&submit=Search%21&idxname=kca&max=100&result=short&sort=date%3Aearly > > > > Here's the result: > > > > 1. win 98SE (score: 2) > > /pipermail/kca/2002-September/000192.html (4,152 bytes) > ... > > You can see from path names that these are out of order. Here are the Date > > fields in each of these, copy-n-pasted from the files themselves: > > > > Sun Sep 22 14:07:51 CDT 2002 > > Fri Jul 19 11:54:00 CDT 2002 > > Fri Aug 23 08:44:57 CDT 2002 > > Mon Mar 3 10:14:12 CST 2003 > > Tue Jan 14 17:37:11 CST 2003 > > To begin with, Date of pipermail was not RFC2822 form. > However, Date of pipermail is correctly reflected in the field 'Date'. The pipermail format contains no RFC822 header, but morphs the "Date" header from the original post into an HTML element, and the date format is different from what was in the original message/rfc822 format. This is apparently correctly parsed by mknmz. For instance, the original message contained: Date: Sun, 22 Sep 2002 14:07:51 -0500 Whereas Mailman's pipermail conversion converts this to: Sun Sep 22 14:07:51 CDT 2002 Namazu must handle this difference. Pipermail files, at least those generated by Mailman, aren't in RFC822 format, nor can the program reasonably expect them to be in this format. > Then, let's examine the time stamp of pipermail file next. > For instance, please confirm the following file and confirm the date > by 'ls -alF'. The Unix time stamp is irrelevant! The entire pipermail archive file hierarchy can be rebuilt using the Mailman "arch" utility (arch --wipe listname) in which case the time stamp on these files will be the time of the rebuild, not the posting time. No indexing utility (or any other utility for that matter) can expect to get valid posting date/time data from the Unix file mod time! > One is a method of changing the time stamp of the file according > to information on Date of contents of the file. No accessory utility such as namazu should ever make any changes to the data source it's analyzing, even if it's only a matter of changing the Unix time stamp. Even if this weren't bad software design, it's a rather excessive and inefficient way to do the job. If namazu can correctly parse out the date from a pipermail html file, as it seems to be able to do in pipermail.pl, then it can certainly store this information in an index so it can be used to sort the files in correct date order. Moreover, although I haven't looked at all the code, nowhere in the code I looked at did it look as if namazu was trying to sequence files based on the Unix time stamp! I'm not sure what you're saying here. Does namazu expect to be able to do date sorts based on the Unix timestamp on pipermail files? I find this hard to believe since the namazu code looks pretty intelligent otherwise. I'll run some tests to see if this is true, but if it is, we'll have to abandon our project to integrate namazu into mailman since this kind of behavior would constitute a serious design flaw. > Another one is a method of using the field sorting UTC function. > Do you know the method of setting the field sorting UTC function? I'm not familiar with the field sorting UTC function. What does namazu use, and where in the code is the sorting done? Are you one of the authors of namazu? -- Lindsay Haisley | "The voice of dissent | PGP public key FMP Computer Services | was arrested before the | available at 512-259-1190 | president cleared his | http://pubkeys.fmp.com http://www.fmp.com | throat to speak | | of freedom" | | (Chris Chandler) | From fmouse at fmp.com Sat Jan 13 08:46:17 2007 From: fmouse at fmp.com (Lindsay Haisley) Date: Sun Jan 14 06:09:50 2007 Subject: [Namazu-users-en] Further note on problems with date sorts Message-ID: <20070112234617.GB11197@fmp.com> I should note that my mknmz invocation is as follows: /usr/bin/mknmz -O $1 --media-type='text/html; x-type=pipermail' \ --config=/home/nmz/$1/.mknmzrc --template-dir=$1 \ /usr/local/mailman/archives/private/$1 $1 is the directory, named for the list name. -- Lindsay Haisley | "Fighting against human | PGP public key FMP Computer Services | creativity is like | available at 512-259-1190 | trying to eradicate | http://www.fmp.com | dandelions" | | (Pamela Jones) | From fmouse at fmp.com Sat Jan 13 11:03:10 2007 From: fmouse at fmp.com (Lindsay Haisley) Date: Sun Jan 14 06:09:51 2007 Subject: [Namazu-users-en] Re: Further note on problems with date sorts In-Reply-To: <20070112234815.GC11197@fmp.com> References: <20070112234815.GC11197@fmp.com> Message-ID: <20070113020310.GD11197@fmp.com> I did a bit of work on this. Looks like pipermail.pl is correctly grabbing the dates from the pipermail files. The problem is elsewhere. Thus spake Lindsay Haisley on Fri, Jan 12, 2007 at 05:48:15PM CST > I should note that my mknmz invocation is as follows: > > /usr/bin/mknmz -O $1 --media-type='text/html; x-type=pipermail' \ > --config=/home/nmz/$1/.mknmzrc --template-dir=$1 \ > /usr/local/mailman/archives/private/$1 > > $1 is the directory, named for the list name. > > -- > Lindsay Haisley | "Fighting against human | PGP public key > FMP Computer Services | creativity is like | available at > 512-259-1190 | trying to eradicate | > http://www.fmp.com | dandelions" | > | (Pamela Jones) | > _______________________________________________ > Namazu-users-en mailing list > Namazu-users-en@namazu.org > http://www.namazu.org/cgi-bin/mailman/listinfo/namazu-users-en -- Lindsay Haisley | "Fighting against human | PGP public key FMP Computer Services | creativity is like | available at 512-259-1190 | trying to eradicate | http://www.fmp.com | dandelions" | | (Pamela Jones) | From yw3t-trns at asahi-net.or.jp Sun Jan 14 11:15:13 2007 From: yw3t-trns at asahi-net.or.jp (Tadamasa Teranishi) Date: Sun Jan 14 11:15:25 2007 Subject: [Namazu-users-en] Re: Problems with date sorts References: <1168711030.5119.6.camel@localhost.localdomain> <20070113181451.GA12872@fmp.com> Message-ID: <45A99231.F4FE8FE2@asahi-net.or.jp> Though there seems to be a misunderstanding.... It is a function to sort the sorting function with the time stamp of the document file at the date of Namazu. This is old to the specification. Lindsay Haisley wrote: > > > To begin with, Date of pipermail was not RFC2822 form. > > However, Date of pipermail is correctly reflected in the field 'Date'. > > The pipermail format contains no RFC822 header, but morphs the "Date" > header from the original post into an HTML element, and the date format > is different from what was in the original message/rfc822 format. This > is apparently correctly parsed by mknmz. For instance, the original > message contained: Yes. Though it becomes a repetition... Date of pipermail was not RFC2822 originally form. However, mknmz(filter/pipermail.pl) correctly reflects Date of pipermail in the Date field. Only when usually displaying it in the retrieval result, the Date field is used. > > Then, let's examine the time stamp of pipermail file next. > > For instance, please confirm the following file and confirm the date > > by 'ls -alF'. > > The Unix time stamp is irrelevant! No. The time stamp of the file is important. Because the date sorting is done based on the time stamp. > > One is a method of changing the time stamp of the file according > > to information on Date of contents of the file. > > No accessory utility such as namazu should ever make any changes to the > data source it's analyzing, even if it's only a matter of changing the > Unix time stamp. It is a difference in outlook. There is mailutime command in the utility of Namazu. mailutime is a tool to set timestamps of Mail/News files to the Date: header. (However, mailutime doesn't correspond to pipermail.) http://www.namazu.org/doc/manual.html.en#mailutime > > Another one is a method of using the field sorting UTC function. > > Do you know the method of setting the field sorting UTC function? > > I'm not familiar with the field sorting UTC function. What does namazu > use, and where in the code is the sorting done? Namazu also provides the function to use the value of the field as a key to sorting. And, sorting by specifying the UTC field for a key to sorting in date displayed in the retrieval result the order becomes possible. Perhaps, the purpose will be able to be accomplished by using the UTC field sorting. Method of using UTC sorting function: 1. "utc" is added to $SEARCH_FIELD of .mknmzrc. 2. NMZ.head is edited. | V 3. It tries to delete the index, and to make it newly. > Are you one of the authors of namazu? Yes. See CREDITS, ChangeLog, etc... * As for tarball since Namazu 2.0.13, what I signed is distributed. * filter/pipermail.pl is what there is a file that became a radical I made. * It is I that added the UTC field sorting function. -- ===================================================================== TADAMASA TERANISHI yw3t-trns@asahi-net.or.jp http://www.asahi-net.or.jp/~yw3t-trns/index.htm Key fingerprint = 474E 4D93 8E97 11F6 662D 8A42 17F5 52F4 10E7 D14E From fmouse-namazu at fmp.com Sun Jan 14 11:54:27 2007 From: fmouse-namazu at fmp.com (Lindsay Haisley) Date: Sun Jan 14 11:54:33 2007 Subject: [Namazu-users-en] Re: Problems with date sorts In-Reply-To: <45A99231.F4FE8FE2@asahi-net.or.jp> References: <1168711030.5119.6.camel@localhost.localdomain> <20070113181451.GA12872@fmp.com> <45A99231.F4FE8FE2@asahi-net.or.jp> Message-ID: <20070114025427.GA15913@fmp.com> Thus spake Tadamasa Teranishi on Sat, Jan 13, 2007 at 08:15:13PM CST > Though there seems to be a misunderstanding.... > > It is a function to sort the sorting function with the time stamp of > the document file at the date of Namazu. > This is old to the specification. Well this gives me something to think about and work with. Thanks. I'll take a bit if time and dig further for the source of the problem. I have a bit of difficulty with your English, as you may have with mine. Sorry ;-) I'll figure it out. The pipermail files on the server here using namazu were transferred from another Mailman installation on another server. All of them have a timestamp on them that is no earlier than the time of this transfer. Some pipermail archive collections were trimmed and regenerated using Mailman's 'arch' utility, and their timestamp shows the time they were regenerated. Furthermore, on my system, mknmz runs as the 'nmz' user which doesn't have write permission on the pipermail files, so it can't change the timestamps. Are you telling me that if I allow mknmz to run with write permission on the Mailman pipermail files, that it will make their Unix timestamps correspond to the posting dates in them and namazu.cgi sorting will work correctly at run time? > Lindsay Haisley wrote: > > > > > To begin with, Date of pipermail was not RFC2822 form. > > > However, Date of pipermail is correctly reflected in the field 'Date'. > > > > The pipermail format contains no RFC822 header, but morphs the "Date" > > header from the original post into an HTML element, and the date format > > is different from what was in the original message/rfc822 format. This > > is apparently correctly parsed by mknmz. For instance, the original > > message contained: > > Yes. Though it becomes a repetition... > Date of pipermail was not RFC2822 originally form. > However, mknmz(filter/pipermail.pl) correctly reflects Date of > pipermail in the Date field. > > Only when usually displaying it in the retrieval result, > the Date field is used. > > > > Then, let's examine the time stamp of pipermail file next. > > > For instance, please confirm the following file and confirm the date > > > by 'ls -alF'. > > > > The Unix time stamp is irrelevant! > > No. > The time stamp of the file is important. > Because the date sorting is done based on the time stamp. > > > > One is a method of changing the time stamp of the file according > > > to information on Date of contents of the file. > > > > No accessory utility such as namazu should ever make any changes to the > > data source it's analyzing, even if it's only a matter of changing the > > Unix time stamp. > > It is a difference in outlook. > > There is mailutime command in the utility of Namazu. > mailutime is a tool to set timestamps of Mail/News files to the Date: > header. > (However, mailutime doesn't correspond to pipermail.) > > http://www.namazu.org/doc/manual.html.en#mailutime > > > > Another one is a method of using the field sorting UTC function. > > > Do you know the method of setting the field sorting UTC function? > > > > I'm not familiar with the field sorting UTC function. What does namazu > > use, and where in the code is the sorting done? > > Namazu also provides the function to use the value of the field as a > key to sorting. > > And, sorting by specifying the UTC field for a key to sorting in date > displayed in the retrieval result the order becomes possible. > > Perhaps, the purpose will be able to be accomplished by using the > UTC field sorting. > > Method of using UTC sorting function: > > 1. "utc" is added to $SEARCH_FIELD of .mknmzrc. > 2. NMZ.head is edited. > > > > > | > V > > > > > 3. It tries to delete the index, and to make it newly. > > > Are you one of the authors of namazu? > > Yes. See CREDITS, ChangeLog, etc... > > * As for tarball since Namazu 2.0.13, what I signed is distributed. > * filter/pipermail.pl is what there is a file that became a radical > I made. > * It is I that added the UTC field sorting function. > -- > ===================================================================== > TADAMASA TERANISHI yw3t-trns@asahi-net.or.jp > http://www.asahi-net.or.jp/~yw3t-trns/index.htm > Key fingerprint = 474E 4D93 8E97 11F6 662D 8A42 17F5 52F4 10E7 D14E > > _______________________________________________ > Namazu-users-en mailing list > Namazu-users-en@namazu.org > http://www.namazu.org/cgi-bin/mailman/listinfo/namazu-users-en -- Lindsay Haisley | "Fighting against human | PGP public key FMP Computer Services | creativity is like | available at 512-259-1190 | trying to eradicate | http://www.fmp.com | dandelions" | | (Pamela Jones) | From yw3t-trns at asahi-net.or.jp Sun Jan 14 12:21:56 2007 From: yw3t-trns at asahi-net.or.jp (Tadamasa Teranishi) Date: Sun Jan 14 12:22:10 2007 Subject: [Namazu-users-en] Re: Problems with date sorts References: <1168711030.5119.6.camel@localhost.localdomain> <20070113181451.GA12872@fmp.com> <45A99231.F4FE8FE2@asahi-net.or.jp> <20070114025427.GA15913@fmp.com> Message-ID: <45A9A1D4.974CF1C9@asahi-net.or.jp> # Stop the full text quotation. Lindsay Haisley wrote: > > Well this gives me something to think about and work with. Thanks. I'll take > a bit if time and dig further for the source of the problem. I have a bit of > difficulty with your English, as you may have with mine. Sorry ;-) I'll > figure it out. I'm sorry because it is not good at English. > on them that is no earlier than the time of this transfer. Some pipermail > archive collections were trimmed and regenerated using Mailman's 'arch' > utility, and their timestamp shows the time they were regenerated. It is likely to be displayed in this time order of the stamp, now. > Furthermore, on my system, mknmz runs as the 'nmz' user which doesn't have > write permission on the pipermail files, so it can't change the timestamps. When the pipermail file is copied, the time stamp will only have to be changed by some methods. (The tool to change the time stamp of the pipermail file is not prepared in Namazu. ) > Are you telling me that if I allow mknmz to run with write permission on the > Mailman pipermail files, that it will make their Unix timestamps correspond to > the posting dates in them and namazu.cgi sorting will work correctly at run > time? No. The user must change the time stamp. However, if the field sorting of the UTC is used without using sorting at a standard date, the time stamp need not be changed. It might be easy and be good to use the UTC field sorting in case of your purpose. -- ===================================================================== TADAMASA TERANISHI yw3t-trns@asahi-net.or.jp http://www.asahi-net.or.jp/~yw3t-trns/index.htm Key fingerprint = 474E 4D93 8E97 11F6 662D 8A42 17F5 52F4 10E7 D14E From yw3t-trns at asahi-net.or.jp Thu Jan 25 00:01:17 2007 From: yw3t-trns at asahi-net.or.jp (Tadamasa Teranishi) Date: Thu Jan 25 00:01:02 2007 Subject: [Namazu-users-en] Namazu for Windows 2.0.16 (archive 003) Message-ID: <45B774BD.E46AC5A0@asahi-net.or.jp> Namazu for Windows 2.0.16 (archive 003) has been opened to the public. (This package is a version for Windows Vista. ) The compiled binary was made about a formal installation package to precede and to use Namazu 2.0.16 with Windows because it was not prepared. http://www.akaneiro.jp/public/nmz2.0.16.003-win32.zip http://www.akaneiro.jp/public/nmz2.0.16.003-win32.zip.sig http://www.akaneiro.jp/public/nmz2.0.16.003-win32.zip.md5 (md5sum: 99795bc118085cce3d77f0debd859903) http://www.akaneiro.jp/public/nmz2.0.16.003-win32.zip.sha1 (sha1sum: 3c81102728236b341f56943bac46fd403ee4df34) Because the installer is not included in this archive to our regret, it is necessary to install it by hand power. The system requirements - It limits it to the ActivePerl 800 series. - The installation directory is limited to C:\namazu. Moreover, when installing it, it is necessary to hook up to the Internet because it is necessary to install PPM file from the Web server. -- ===================================================================== TADAMASA TERANISHI yw3t-trns@asahi-net.or.jp http://www.asahi-net.or.jp/~yw3t-trns/index.htm Key fingerprint = 474E 4D93 8E97 11F6 662D 8A42 17F5 52F4 10E7 D14E From latelate at literaturlatenight.de Fri Jan 26 15:44:39 2007 From: latelate at literaturlatenight.de (harry meyers) Date: Fri Jan 26 15:49:49 2007 Subject: [Namazu-users-en] Viewing search results Message-ID: <87ac069tig.fsf@literaturlatenight.de> When I do a bknamazu -b firefox some.file I can see the current list (1-20) but - if the results are more than twenty - I cannot flip through the pages, only the first 20 results are shown. It makes no difference which browser I use. Only within emacs I got the following pages listed. Regard harry From yw3t-trns at asahi-net.or.jp Fri Jan 26 16:21:03 2007 From: yw3t-trns at asahi-net.or.jp (Tadamasa Teranishi) Date: Fri Jan 26 16:19:16 2007 Subject: [Namazu-users-en] Re: Viewing search results References: <87ac069tig.fsf@literaturlatenight.de> Message-ID: <45B9ABDF.AF854621@asahi-net.or.jp> harry meyers wrote: > > bknamazu -b firefox some.file Is it a mistake of bnamazu? > I can see the current list (1-20) but - if the results are > more than twenty - I cannot flip through the pages, only the > first 20 results are shown. It makes no difference which $ bnamazu Usage: bnamazu [-n] [-b browser] [namazu's options] [index]... -b browser set your faborite browser for browsing. [lynx] -n open a new window when you browse with netscape. Notes: Other options will be passed to namazu. The option of namazu can be specified. $ namazu --help namazu 2.0.16, a search program of Namazu. Usage: namazu [options] [index]... -n, --max=NUM set the number of documents shown to NUM. -w, --whence=NUM set the first number of documents shown to NUM. -l, --list print the results by listing the format. -s, --short print the results in a short format. --result=EXT set NMZ.result.EXT for printing the results. --late sort the documents in late order. --early sort the documents in early order. --sort=METHOD set a sort METHOD (score, date, field:name) --ascending sort in ascending order (default: descending) -a, --all print all results. : : * If -a option is specified, everything can be displayed. ex) bnamazu -a query index * If the display beginning number is specified by -w option, it can display since the 21st. ex) bnamazu -w 20 query index * If the display number is specified by --max option, 20 or more can be displayed. ex) bnamazu --max=50 query index Please do not use -n of a short option to overlap with the option of bnamazu. -- ===================================================================== TADAMASA TERANISHI yw3t-trns@asahi-net.or.jp http://www.asahi-net.or.jp/~yw3t-trns/index.htm Key fingerprint = 474E 4D93 8E97 11F6 662D 8A42 17F5 52F4 10E7 D14E From latelate at literaturlatenight.de Fri Jan 26 22:27:01 2007 From: latelate at literaturlatenight.de (harry meyers) Date: Fri Jan 26 22:27:12 2007 Subject: [Namazu-users-en] Re: Viewing search results In-Reply-To: <45B9ABDF.AF854621@asahi-net.or.jp> (Tadamasa Teranishi's message of "Fri\, 26 Jan 2007 16\:21\:03 +0900") References: <87ac069tig.fsf@literaturlatenight.de> <45B9ABDF.AF854621@asahi-net.or.jp> Message-ID: <87fy9x536i.fsf@literaturlatenight.de> Tadamasa Teranishi writes: > harry meyers wrote: >> bknamazu -b firefox some.file > > Is it a mistake of bnamazu? > Ah, sorry, that's only a typing error > * If -a option is specified, everything can be displayed. ex) bnamazu > -a query index > > * If the display beginning number is specified by -w option, it can > display since the 21st. ex) bnamazu -w 20 query index > > * If the display number is specified by --max option, 20 or more can > be displayed. ex) bnamazu --max=50 query index Okay, bnamazu with the -a option lets me show all results in a huge page. But if I remember correctly once it was possible to show the results in a browser window similiar to this (copied from emacs): Current List: 1 - 30 Page: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18] If I remember correctly again this was the output even if started without any other option as -b for the browser. harry From yw3t-trns at asahi-net.or.jp Fri Jan 26 22:51:09 2007 From: yw3t-trns at asahi-net.or.jp (Tadamasa Teranishi) Date: Fri Jan 26 22:49:20 2007 Subject: [Namazu-users-en] Re: Viewing search results References: <87ac069tig.fsf@literaturlatenight.de> <45B9ABDF.AF854621@asahi-net.or.jp> <87fy9x536i.fsf@literaturlatenight.de> Message-ID: <45BA074D.21546E90@asahi-net.or.jp> harry meyers wrote: > > Current List: 1 - 30 > Page: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18] > > If I remember correctly again this was the output even if > started without any other option as -b for the browser. It might be a misunderstanding because it cannot use it in namazu and bnamazu. It is possible only if it displays it by "-H" option. However, the link is not effective. Because not namazu but namazu.cgi makes this function effective. -- ===================================================================== TADAMASA TERANISHI yw3t-trns@asahi-net.or.jp http://www.asahi-net.or.jp/~yw3t-trns/index.htm Key fingerprint = 474E 4D93 8E97 11F6 662D 8A42 17F5 52F4 10E7 D14E