Page 1 of 2

How do I flush a pipe ?

Posted: Sat Aug 21, 2010 9:55 pm
by Matthias Gemuh
How do I flush a pipe under Windows ?

My GUI talks to child processes using pipes. I write into them using WriteFile(Handle, ...);
How can I flush such output pipes ?

Matthias.

Re: How do I flush a pipe ?

Posted: Sat Aug 21, 2010 11:11 pm
by kingliveson
Have you explored fflush?

Re: How do I flush a pipe ?

Posted: Sun Aug 22, 2010 10:47 am
by Matthias Gemuh
kingliveson wrote:Have you explored fflush?
I casted a handle to a stream pointer to use with fflush(), but it did not work.

Anyway, HGM says a pipe cannot be flushed.

Matthias.

Re: How do I flush a pipe ?

Posted: Sun Aug 22, 2010 4:39 pm
by hyatt
Matthias Gemuh wrote:
kingliveson wrote:Have you explored fflush?
I casted a handle to a stream pointer to use with fflush(), but it did not work.

Anyway, HGM says a pipe cannot be flushed.

Matthias.
The "pipe" part is irrelevant. The "buffer" is in the C library code. That's where you get into trouble. Easiest way to deal with this is to simply use read()/write() as I do in Crafty. They are, by standard, unbuffered. Which means that as soon as you do a write, the data is "on its way" to the final destination. It doesn't sit in a C library buffer until the buffer is filled, before it is sent. The engine-intf.html document explains that in detail.

Re: How do I flush a pipe ?

Posted: Sun Aug 22, 2010 5:22 pm
by kingliveson
hyatt wrote:
Matthias Gemuh wrote:
kingliveson wrote:Have you explored fflush?
I casted a handle to a stream pointer to use with fflush(), but it did not work.

Anyway, HGM says a pipe cannot be flushed.

Matthias.
The "pipe" part is irrelevant. The "buffer" is in the C library code. That's where you get into trouble. Easiest way to deal with this is to simply use read()/write() as I do in Crafty. They are, by standard, unbuffered. Which means that as soon as you do a write, the data is "on its way" to the final destination. It doesn't sit in a C library buffer until the buffer is filled, before it is sent. The engine-intf.html document explains that in detail.
Ok, I just read what HGM said and don't think he is saying a pipe cannot be flushed. I think as Prof. Hyatt pointed out, you are not targeting the key area, the buffer. Using read()/write() is probably best trying to redirect output in this case, as it is more efficient -- if I understand correctly what you're trying to do . See MSDN FlushFileBuffers note below:
Typically the WriteFile and WriteFileEx functions write data to an internal buffer that the operating system writes to a disk or communication pipe on a regular basis. The FlushFileBuffers function writes all the buffered information for a specified file to the device or pipe.

Due to disk caching interactions within the system, the FlushFileBuffers function can be inefficient when used after every write to a disk drive device when many writes are being performed separately. If an application is performing multiple writes to disk and also needs to ensure critical data is written to persistent media, the application should use unbuffered I/O instead of frequently calling FlushFileBuffers. To open a file for unbuffered I/O, call the CreateFile function with the FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH flags. This prevents the file contents from being cached and flushes the metadata to disk with each write. For more information, see CreateFile.

Re: How do I flush a pipe ?

Posted: Sun Aug 22, 2010 7:09 pm
by Matthias Gemuh
kingliveson wrote: Typically the WriteFile and WriteFileEx functions write data to an internal buffer that the operating system writes to a disk or communication pipe on a regular basis. The FlushFileBuffers function writes all the buffered information for a specified file to the device or pipe.

Due to disk caching interactions within the system, the FlushFileBuffers function can be inefficient when used after every write to a disk drive device when many writes are being performed separately. If an application is performing multiple writes to disk and also needs to ensure critical data is written to persistent media, the application should use unbuffered I/O instead of frequently calling FlushFileBuffers. To open a file for unbuffered I/O, call the CreateFile function with the FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH flags. This prevents the file contents from being cached and flushes the metadata to disk with each write. For more information, see CreateFile.
I have been using WriteFile() without knowing that it buffers.
That is why I was suspecting the buffer to be in the pipe. :evil:

Re: How do I flush a pipe ?

Posted: Mon Aug 23, 2010 9:22 pm
by kranium
1st and fore all: try the good old fashioned plunger! (this age-old remedy can often do wonders)

Drano (and/or Mr. Plumber) is quite good...but excercise caution and read the directions carefully if you have PVC.

otherwise: i highly recommend the help of an experienced professional...especially if the pipe is lead.

hope this helps...
Norm

Re: How do I flush a pipe ?

Posted: Mon Aug 23, 2010 9:40 pm
by Matthias Gemuh
kranium wrote:1st and fore all: try the good old fashioned plunger! (this age-old remedy can often do wonders)

Drano (and/or Mr. Plumber) is quite good...but excercise caution and read the directions carefully if you have PVC.

otherwise: i highly recommend the help of an experienced professional...especially if the pipe is lead.

hope this helps...
Norm
What has a plumber got to do with what I smoke ? :P

Matthias.

Re: How do I flush a pipe ?

Posted: Mon Aug 23, 2010 10:27 pm
by benstoker
Matthias Gemuh wrote:How do I flush a pipe under Windows ?

My GUI talks to child processes using pipes. I write into them using WriteFile(Handle, ...);
How can I flush such output pipes ?

Matthias.
http://www.pipesandcigars.com/pipecleaners.html

Re: How do I flush a pipe ?

Posted: Thu Aug 26, 2010 9:45 pm
by kingliveson
From CCC:
Matthias Gemuh wrote:One last trial to explain it.

Windows offers only WriteFile() to write into a pipe.
(Obselete write functions do exist that work like WriteFile())
WriteFile() has an internal buffer that makes WriteFile() non-blocking.
Windows flushes that buffer depending on how full it is or how long data has been standing in it. This efficiency decision effectively steals time from any engine waiting at the other end of the pipe.
The internal WriteFile() buffer can be forcefully flushed with FlushFileBuffers() to eliminate delays, but that is blocking
You can accomplish read()/write() effect using FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH flags with WriteFile().