Welcome to MacBoardz.com!
FAQFAQ    SearchSearch      ProfileProfile    Private MessagesPrivate Messages   Log inLog in

readdir() acts funny

 
   Macintosh computer (Home) -> General Discussion RSS
Next:  link two spreadsheets  
Author Message
Jedrzej Dudkiewicz

External


Since: Jun 29, 2005
Posts: 8



(Msg. 1) Posted: Wed Mar 15, 2006 4:55 pm
Post subject: readdir() acts funny
Archived from groups: comp>sys>mac>misc (more info?)

Hello.

I wrote a simple disk scanner. It is supposed to replace `find` since one
can't force `find` to skip directories with given names. It is required, as
there are humongous directories with backup files and going through them
takes ages (they are on NFS volumes, which also must be scanned).

So I wrote this program, and run it on few Unix systems. Namely Fedora Core
3, Solaris 5.7, 5.9 and 5.10, AIX 4 and 5, Tru64, HPUX 10.20 and 11.11. So I
thought that everything will be fine on Mac OS X. I've started with 10.4,
leaving 1, 2 and 3 for "after succesfuly completed tests on 10.4."

And now I'm stucked. Issuing command:

find / -type f

results in about 4.7MB log (it's still not enough, check end of mail for
details). Scanning with my program, which is supposed to do the same
thing... never ends. I were able to create 7GB of log and my prog seems to
be stuck for some reason, like following symlinks. But I'm not.

My programs scans directories recursively, and hm... eagerly. That is:

scan(path) {
opendir(path);
while(readdir()) {
lstat(file);
if(regularfile)
printf(filename);
if(directory)
scan(filename);
// ignore all other types
}
}

It looks like first directory stream, that is one that is created for /
path, is running in loop! I've logged how it works, with "depth of call",
and same directories appear on same level forever and again.

I changed the program, so now it scans directory and saves names of entries
found and then checks whether given name is directory or regular file and
acts accordingly. It works now, but takes too much memory. I can reduce this
usage by saving only directories names, but still it seems like a very bad
idea.

What more, `find` utility scans only first three directories, and than
writes:
find: /dev/fd/4: No such file or directory
find: /Developer: No such file or directory
find: /encrypting: No such file or directory
find: /googlesearchwidget.zip.download: No such file or directory
find: /jd: No such file or directory
find: /Library: No such file or directory
find: /mnt: No such file or directory
find: /Network: No such file or directory
[ list goes on ]

My modified program scans all drives (nfs and hfs) and produces about 30MB
of logs. It seems that find has some problems too.

So, finally, the cry for help: HELP! Smile

If someone have seen and defeated such a beast, please share your solution.

Thanks in advance,

JD

 >> Stay informed about: readdir() acts funny 
Back to top
Login to vote
Jedrzej Dudkiewicz

External


Since: Jun 29, 2005
Posts: 8



(Msg. 2) Posted: Wed Mar 15, 2006 4:55 pm
Post subject: Re: readdir() acts funny [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Please forgive me for some terrible mistakes. It's 10PM in Poland and I'm
working for 14 hours now and I'm not exactly in a best shape to write long
posts in foreign languages :/

JD

 >> Stay informed about: readdir() acts funny 
Back to top
Login to vote
user638

External


Since: Jan 13, 2005
Posts: 633



(Msg. 3) Posted: Wed Mar 15, 2006 11:55 pm
Post subject: Re: readdir() acts funny [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

In article ,
"Jedrzej Dudkiewicz"
wrote:

> Hello.
>
> I wrote a simple disk scanner. It is supposed to replace `find` since one
> can't force `find` to skip directories with given names. It is required, as
> there are humongous directories with backup files and going through them
> takes ages (they are on NFS volumes, which also must be scanned).
>
> So I wrote this program, and run it on few Unix systems. Namely Fedora Core
> 3, Solaris 5.7, 5.9 and 5.10, AIX 4 and 5, Tru64, HPUX 10.20 and 11.11. So I
> thought that everything will be fine on Mac OS X. I've started with 10.4,
> leaving 1, 2 and 3 for "after succesfuly completed tests on 10.4."
>
> And now I'm stucked. Issuing command:
>
> find / -type f
>
> results in about 4.7MB log (it's still not enough, check end of mail for
> details). Scanning with my program, which is supposed to do the same
> thing... never ends. I were able to create 7GB of log and my prog seems to
> be stuck for some reason, like following symlinks. But I'm not.
>
> My programs scans directories recursively, and hm... eagerly. That is:
>
> scan(path) {
> opendir(path);
> while(readdir()) {
> lstat(file);
> if(regularfile)
> printf(filename);
> if(directory)
> scan(filename);
> // ignore all other types
> }
> }
>
> It looks like first directory stream, that is one that is created for /
> path, is running in loop! I've logged how it works, with "depth of call",
> and same directories appear on same level forever and again.
>
> I changed the program, so now it scans directory and saves names of entries
> found and then checks whether given name is directory or regular file and
> acts accordingly. It works now, but takes too much memory. I can reduce this
> usage by saving only directories names, but still it seems like a very bad
> idea.
>
> What more, `find` utility scans only first three directories, and than
> writes:
> find: /dev/fd/4: No such file or directory
> find: /Developer: No such file or directory
> find: /encrypting: No such file or directory
> find: /googlesearchwidget.zip.download: No such file or directory
> find: /jd: No such file or directory
> find: /Library: No such file or directory
> find: /mnt: No such file or directory
> find: /Network: No such file or directory
> [ list goes on ]
>
> My modified program scans all drives (nfs and hfs) and produces about 30MB
> of logs. It seems that find has some problems too.
>
> So, finally, the cry for help: HELP! Smile
>
> If someone have seen and defeated such a beast, please share your solution.
>
> Thanks in advance,
>
> JD

I'm not sure about the "No such file or directory" 'find' errors.
But this trick might work to keep 'find' from walking directories
that match specific patterns:

find /starting/path ! -name "dir_name_to_exclude*" -o -prune

What this says is if the file name does not match
"dir_name_to_exclude*", then output that name. If the name
matches "dir_name_to_exclude*" then the -name fails, so the -o
(OR) is used, and the -prune prevents 'find' from processing the
directory.

Depending on what you want to do, you may be able to create some
complex logical test using -a (AND), -o (OR), and parens \( ... \)
expressions to cause -prune to be used at the right time, and
otherwise just the files you are interested in to be returned.

find /path \( -type f \) -o \
\( -type d -a ! -name "*exclude*" -o -prune)

Or maybe some cascading piped finds, like:

find /path -type d -a ! -name "*exclude*" -o -prune | \
xargs -J % find % -type f -a -name "*file_select*"

I do not know if this will be helpful or not, but it might pass
along some ideas for you to think about.

Bob Harris
 >> Stay informed about: readdir() acts funny 
Back to top
Login to vote
Jedrzej Dudkiewicz

External


Since: Jun 29, 2005
Posts: 8



(Msg. 4) Posted: Thu Mar 16, 2006 5:55 am
Post subject: Re: readdir() acts funny [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

> I'm not sure about the "No such file or directory" 'find' errors.
> But this trick might work to keep 'find' from walking directories
> that match specific patterns:
>
> find /starting/path ! -name "dir_name_to_exclude*" -o -prune
>
> What this says is if the file name does not match
> "dir_name_to_exclude*", then output that name. If the name
> matches "dir_name_to_exclude*" then the -name fails, so the -o
> (OR) is used, and the -prune prevents 'find' from processing the
> directory.

Thanks for help. I'll check it. Never the less, it seems strange, that
readdir fails in such a miserable way.

JD
 >> Stay informed about: readdir() acts funny 
Back to top
Login to vote
Jedrzej Dudkiewicz

External


Since: Jun 29, 2005
Posts: 8



(Msg. 5) Posted: Fri Mar 17, 2006 5:55 am
Post subject: Re: readdir() acts funny [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

> Depending on what you want to do, you may be able to create some
> complex logical test using -a (AND), -o (OR), and parens \( ... \)
> expressions to cause -prune to be used at the right time, and
> otherwise just the files you are interested in to be returned.

Well, it appears that one can include/exclude almost anything using this
operators, but my list of directories to skip can change, and it looks like
I'd have to write really complicated(?) piece of code using shell script to
create such "find query." So I'll stick with binary, although it required
really weird actions.

JD
 >> Stay informed about: readdir() acts funny 
Back to top
Login to vote
Display posts from previous:   
Related Topics:
Power Macintosh acts up - Good day, Guten Tag! I use a Powermac with 400mHz, around 160 MB RAM and System 8.6. Recently this machine has behaved oddly, for example: 1) It freezes, commonly if I click one of the menus, in Netscape or Mozilla, and in Finder. 2) Odd files..

adobe acrobat acts weird with regard to commenting - lets say that I put comments into and adobe reader document, then I quit for the day. next time I come back the comments are nowhere to be found. ???? this is not a very funny joke adobe is playing

Funny one - I keep telling my Mail program that Apple mail is "Junk" and Apple's mail keeps showing up in my Inbox, not the Junk folder. Lee Park Ridge, NJ rarebirdyatverizondotnet

InterfaceLib..GetComponentResource - Hi All! Bet you've seen this FRUSTRATING one before... I cannot open many applications because of this ingredient which has mysteriously disappeared from my iMac 233MHz running OS9.1. It seems to have something to do with Quicktime, and is apparently..

building your own mac - Hello everyone. I've got another noob question. Is it possible to buy your own Mac hardware and assemble the computer yourself, to save on the cost of buying a factory machine? -Dan
   Macintosh computer (Home) -> General Discussion All times are: Pacific Time (US & Canada)
Page 1 of 1

 
You can post new topics in this forum
You can reply to topics in this forum
You can edit your posts in this forum
You can delete your posts in this forum
You can vote in polls in this forum



[ Contact us | Terms of Service/Privacy Policy ]