pipe() read(), write() - more then 128Kb of data

This is a discussion on pipe() read(), write() - more then 128Kb of data within the C Programming forums, part of the General Programming Boards category; Hello all,
Thanking in advance for the answers again!
I have a program reads from a file then pipes it ...

pipe() read(), write() - more then 128Kb of data

I have a program reads from a file then pipes it to a Unix filter program, this last program adds lines to the original file, then prints it on the screen.

I tested my program with a 116K file, it worked, but then I tried a larger file (400+ Kb) and it halted. The program froze, didn't do anything, as if it's blocking.I found out that a 127-128 KB file will still work, but a larger file will not.

What is causing this?? Is it because I somehow have to manage the reads and writes?

The problem is that, in Child(), you are writing a lot of data to Y[1], but you are not attempting to read from the other end of the pipe (in Parent()) until after the parent has sent all data. However, pipes have a capacity, and the output from Child() is filling up the pipe to its capacity. Once it hits capacity, write() in the child will block. Parent(), however, is still trying to write() to a pipe owned by the child. Since the child is blocking on its write, it will not be able to issue a read() on that pipe, causing Parent() to hit the capacity and block on its write(). You wind up with both parent and child blocking in their respective write() calls because the other side is not issuing any read()s.

- set pipes to non_block
Reason: Parent can writes and the child read at the same time

- Child stores the read data into an array

- Exec() reads the variable as an argument. (normally it would look for the file on the harddisk.)

On top of this: I have to open my files in Binary mode. Reason: Portable code. Linux and windows use different END OF FILE characters. This will cause a read to stop as it ancounters a EOF character. In Binary it would just treat it as a normal character.