[Namazu-users-en] Re: meta-information on large binary files

Tadamasa Teranishi yw3t-trns at asahi-net.or.jp
Tue Jul 11 22:31:34 JST 2006


Alexander Oelzant wrote:
> 
> > If the target files are only one media-type, you can do it like following:
> >
> > $ mknmz -O indexdir -t audio/mpeg target-dir
> >
> > -t (--media-type) option ommits to read target file for finding binary
> > signature.
> 
> Thanks, but unfortunately for indexing the filter still has to read in
> the entire file, for a 200M-file that produces processes like the
> following:

Namazu reads the entire file once. 
In the worst case, it copies with mknmz once, it reads once, it writes 
with the filter once, and it reads once. 

The time that hangs to the presumption of the media type is only 
shortened when -t option is specified. 

> I was hoping the $ON_MEMORY_MAX   = 5000000; would take care of that, e.
> g. only reading in the first part of the file, but according to
> tips.html that only influences the size of the db files kept in memory,

$ON_MEMORY_MAX is not specification s as for the read size of the 
file. 

When two or more files are read, and the size of the read file 
exceeds $ON_MEMORY_MAX, the index is written once. 

"$ON_MEMORY_MAX   = 5000000;" it Every time, it writes it to the 
index at the file of 200MB. 

Because it takes time to the writing processing of the index, 
the frequency is decreased when $ON_MEMORY_MAX is adjusted to a very
big value, and time can be shortened. 

Processing will quicken more than now because it assumes, 
"$ON_MEMORY_MAX   = 1073741824;" if the memory is piled up enough. 
-- 
=====================================================================
TADAMASA TERANISHI  yw3t-trns �� asahi-net.or.jp
http://www.asahi-net.or.jp/~yw3t-trns/index.htm
Key fingerprint =  474E 4D93 8E97 11F6 662D  8A42 17F5 52F4 10E7 D14E



More information about the Namazu-users-en mailing list