You can just do forfin * (or other shell glob), unless you need find’s fancy search/filtering features.
The shell glob isn’t just simpler, but also more robust, because it works also when the filename contains a newline; find .. | whileread -r will crap out on that. Also apparently you want while IFS= read -r because otherwise read might trim whitespace.
If you want to avoid that problem with the newline and still use find, you can use find -exec or find -print0 .. | xargs -0, or find -print0 .. | while IFS= read -r -d ''. I think -print0 is not standard POSIX though.
Still, that was a speed bump that I guess everyone dealing with mass compressed log files has to figure out on the fly because zcat can’t read uncompressed files ! argg !!!
for i in $(ls); do zcat $i 2>/dev/null || cat$i; done | grep mysearchterm
Yeah, it’s a pain. Leads to bad one liners:
for i in $(ls); do zcat $i || cat $i; doneBtw, don’t parse ls. Use
find |while read -rinstead.find -maxdepth 1 -name "term" -print |while read -r file do zcat "$file" 2>/dev/null || cat "$file" doneWon’t this cause cat to iterate through all files in the cwd once zcat encounters an issue, instead of just the specific file?
Yeah, i was tired and had $file there first, then saw that you wanted to cat all in directory. Still tired, but i think this works now.
You are correct. This probably produces something more similar to what you’d want the original command to do, but with better safely:
find -- . -type f -regex '^\./[^/]*$' -exec sh -c -- 'for file in "${@}"; do zcat "${file}" || cat "${file}" || exit; done' sh '{}' '+'That assumes you want to interact with files with names like
.hidden.txt.gzthough. If you don’t, and only intend to have a directory with regular files (as opposed to directories or symbolic links or other types of file), using this is much simpler and even safer, and avoids using files in a surprising order:for i in *; do zcat -- "$i" || cat -- "$i" || exit; doneOf course, the real solution is to avoid using the Shell Command Language at all, and to carefully adapt any program to your particular problem as needed: https://sipb.mit.edu/doc/safe-shell/
You can just do
for f in *(or other shell glob), unless you needfind’s fancy search/filtering features.The shell glob isn’t just simpler, but also more robust, because it works also when the filename contains a newline;
find .. | while read -rwill crap out on that. Also apparently you wantwhile IFS= read -rbecause otherwise read might trim whitespace.If you want to avoid that problem with the newline and still use find, you can use
find -execorfind -print0 .. | xargs -0, orfind -print0 .. | while IFS= read -r -d ''. I think-print0is not standard POSIX though.Doesn’t that depend on the shell?
I don’t think so and have never heard that, but I could be wrong.
Thanks !
But still we shouldn’t have to resort to this !
Also, can’t get the output through pipefor i in $(ls); do zcat $i || cat $i; done | grep mysearchtermthis appears to workfind . -type f -print0 | xargs -0 -I{} sh -c 'zcat "{}" 2>/dev/null || cat "{}"' | grep "mysearchterm"Still, that was a speed bump that I guess everyone dealing with mass compressed log files has to figure out on the fly because zcat can’t read uncompressed files ! argg !!!for i in $(ls); do zcat $i 2>/dev/null || cat $i; done | grep mysearchterm