Author Topic: C: read() and write().  (Read 6303 times)

0 Members and 1 Guest are viewing this topic.

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6130
  • Country: fi
    • My home page and email address
Re: C: read() and write().
« Reply #25 on: March 19, 2021, 01:45:25 pm »
The one thing surprising thing I did learn is that file pointers (that point to where data will be read and written) are by default shared between parent and child processes when you fork(). Found that out pretty quick!
Oh yes, I learned that a while ago, too. Fun fact: those file descriptors (not pointers) are also inherited across an "exec", unless you explicitly open them CLOEXEC. Mighty convenient for inter-process communication, or a gaping security hole waiting to be exploited.
That's why, uh, a friend of mine, recently showed a function creating close-on-exec pipes where neither endpoint is a standard stream, and another function to fork and exec child processes using such pipes and an additional close-on-exec pipe to pass any exec errors back to the parent.  Makes complex pipe trees and nets much easier to construct.  (CC0-1.0 in the hopes of getting people to use something like that in their own code.)
 
The following users thanked this post: DiTBho

Offline madires

  • Super Contributor
  • ***
  • Posts: 7673
  • Country: de
  • A qualified hobbyist ;)
Re: C: read() and write().
« Reply #26 on: March 19, 2021, 02:11:16 pm »
a) check for EINTR and retry?

b) check for things like short writes, and call write() again?

I do both of these, but some people don't believe it is needed, and some wave it away saying they use and trust SA_RESTART)...

Usually I check the results of file functions, but the details depend on the application, i.e. which specific problems need to be tracked and how to react to them.
 

Offline TheCalligrapher

  • Regular Contributor
  • *
  • Posts: 151
  • Country: us
Re: C: read() and write().
« Reply #27 on: April 21, 2021, 05:42:11 pm »
b) check for things like short writes, and call write() again?
I do both of these, but some people don't believe it is needed, and some wave it away saying they use and trust SA_RESTART)...

Not sure how those "some people" might believe it is not needed, considering that the situation is very easy to reproduce. Take a `read` that reads from standard input tied to the console. Input a few characters from keyboard and hit Ctrl+D. Here you go: you have a classic short read. If the programmer fails to handle such situations, their code simply won't work properly.
 

Offline SuntUnMorcov

  • Contributor
  • Posts: 17
  • Country: ro
Re: C: read() and write().
« Reply #28 on: April 29, 2021, 07:13:25 pm »
Not sure how those "some people" might believe it is not needed, considering that the situation is very easy to reproduce. Take a `read` that reads from standard input tied to the console. Input a few characters from keyboard and hit Ctrl+D. Here you go: you have a classic short read. If the programmer fails to handle such situations, their code simply won't work properly.

Ignorance.  Ignorance is bliss.  And if not ignorance, then stubbornness and/or arrogance.  But only one of those qualities can be forgiven and only some of the time.

I've seen code in a real-time application that didn't even check the return code of arguably the most important function call within the whole damn product.  And the criticality of this product failing in production is severe.

Poor quality code is not a rarity and asking why it exists is akin to asking why some people eat Tide pods and why others put their heads into operating thresher machines.
 

Online ataradov

  • Super Contributor
  • ***
  • Posts: 11219
  • Country: us
    • Personal site
Re: C: read() and write().
« Reply #29 on: April 29, 2021, 07:59:46 pm »
I don't think anybody argues that you should not handle errors at all. But simply exiting on incomplete read is good enough handling of a situation, unless application has a specific requirement to handle situations like this.
Alex
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 6769
  • Country: va
Re: C: read() and write().
« Reply #30 on: April 29, 2021, 09:03:59 pm »
Quote
I've seen code in a real-time application that didn't even check the return code of arguably the most important function call within the whole damn product.

Sounds terrible, but you've not given us much to go on. Perhaps the developer decided there was nothing that could be done, given the real-time constraints, regardless of the return code, so no point in checking it.

Normally, and in general, one would expect that read/write return codes are checked and appropriate action taken but, sometimes, "rules are for the guidance of wise men and the obedience of fools."
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14230
  • Country: fr
Re: C: read() and write().
« Reply #31 on: April 29, 2021, 09:09:34 pm »
Quote
I've seen code in a real-time application that didn't even check the return code of arguably the most important function call within the whole damn product.

Sounds terrible, but you've not given us much to go on. Perhaps the developer decided there was nothing that could be done, given the real-time constraints, regardless of the return code, so no point in checking it.

Yeah. I much too often hear that argument, while it can rarely be justified. Unless said function failing is harmless, there is almost ALWAYS something that can be done to mitigate the fault. The only reason for not checking error codes would be if failure would have no, or very minor consequences.
 

Offline thinkfat

  • Supporter
  • ****
  • Posts: 2149
  • Country: de
  • This is just a hobby I spend too much time on.
    • Matthias' Hackerstübchen
Re: C: read() and write().
« Reply #32 on: April 29, 2021, 09:28:12 pm »
Like, are you using the return value of a read() as an index into a buffer without checking.
Everybody likes gadgets. Until they try to make them.
 

Offline SuntUnMorcov

  • Contributor
  • Posts: 17
  • Country: ro
Re: C: read() and write().
« Reply #33 on: April 29, 2021, 10:03:43 pm »
Quote
I've seen code in a real-time application that didn't even check the return code of arguably the most important function call within the whole damn product.

Sounds terrible, but you've not given us much to go on. Perhaps the developer decided there was nothing that could be done, given the real-time constraints, regardless of the return code, so no point in checking it.

Normally, and in general, one would expect that read/write return codes are checked and appropriate action taken but, sometimes, "rules are for the guidance of wise men and the obedience of fools."

I was vague, in this instance, for a reason.  But to clarify, not checking the return code from this particular function call was absolutely inexcusable in this situation.  You absolutely *need* to know if there is an error from this function call because you *need* to take corrective action *immediately*.  In this instance, the choice of the developer was wrong (and mostly caused by ignorance, by their own admission).

And I agree with the sentiment of your point and quote, because that's routed in the exact point I'm trying to make.  The wise men are exactly that because they have enough knowledge, experience, and intuition to make informed decisions.  The fools are exactly that because of their ignorance, stubbornness, and sometimes arrogance.  We can all be both wise men and fools at times (hell, I know I can) and that has an impact on the code people produce.
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 6769
  • Country: va
Re: C: read() and write().
« Reply #34 on: April 30, 2021, 12:05:15 am »
Like, are you using the return value of a read() as an index into a buffer without checking.

A TV recorder of great reputation I have is suspected of doing that. It's very good, unless the signal is a bit poor, and then after arbitary time it would crash. I also noticed some other STBs having similar issues.

As it happens, a bit after I acquired the DVR (and got annoyed with that aspect), I was involved in making a CCTV DVR and got hold of reference hardware and software from an appropriate chip manufacturer. Yep, the example code would take the incoming packet and read off the data length from the header, then blat that amount of data into a buffer. No check that the packet was pukka, or that the length field was meaningfully less than the buffer size. I suspect a fair number of STB badgers copied similar reference code as a quick route to market.
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 6769
  • Country: va
Re: C: read() and write().
« Reply #35 on: April 30, 2021, 12:07:15 am »
Quote
You absolutely *need* to know if there is an error from this function call because you *need* to take corrective action *immediately*.

Fair enough :)
 

Offline TheCalligrapher

  • Regular Contributor
  • *
  • Posts: 151
  • Country: us
Re: C: read() and write().
« Reply #36 on: April 30, 2021, 06:09:57 am »
I don't think anybody argues that you should not handle errors at all. But simply exiting on incomplete read is good enough handling of a situation, unless application has a specific requirement to handle situations like this.

I'm not sure what you are trying to say by this. A spurious short read or a short write is not an error at all. It is perfectly normal behavior of `read` and `write`. Why would anyone even think of "exiting" in response to such situation is beyond me.

This is a fundamental difference between POSIX `read`/`write` and C standard library `fread`/`fwrite`. The latter are not allowed to perform a short read/short write spuriously. With the latter an incomplete read or write is indeed an indication of an error.

But that is not the case with POSIX `read`/`write`. These functions are explicitly allowed to perform a short read/short write "for no reason", as part of their normal behavior. This peculiarity of their behavior has to be taken into account when writing code in terms of `read`/`write`.
 

Online ataradov

  • Super Contributor
  • ***
  • Posts: 11219
  • Country: us
    • Personal site
Re: C: read() and write().
« Reply #37 on: April 30, 2021, 06:17:45 am »
I'm not sure what you are trying to say by this. A spurious short read or a short write is not an error at all. It is perfectly normal behavior of `read` and `write`. Why would anyone even think of "exiting" in response to such situation is beyond me.
Because it does not actually happen on modern OSes outside of direct user input (that Ctrl-D stuff) and other extreme scenarios. It is generally not worth handling that.

Just like it is not worth trying to do something useful if malloc() return NULL. I simply don't care and just exit in that situation.
Alex
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14230
  • Country: fr
Re: C: read() and write().
« Reply #38 on: May 01, 2021, 05:52:39 pm »
There is a difference between not checking at all and just "exiting" if an error occurs.

If you just "exit in that situation", that means you have at least checked. Note that some people will even argue that checking for NULL for malloc() is also useless, as, as your argument is, "on modern OSs", dereferencing NULL will trigger an access violation and exit the program anyway. So the motto of this category of developers is: "just let it crash". Seriously.

This sounds all very yucky to me.

Of course if you're mostly writing non-critical tools, that behavior is alright. As I said earlier, it has no consequence, or only very minor.

This is unacceptable for anything critical though. And as I said earlier, when we claim "there is no way we can handle this case anyway", most of the time, it is simply not true. It's often plain laziness mixed with some bad faith. There are many many examples of this of course. When we were talking about the Boeing 737MAX issue, I remember some also claimed there was nothing that could be done in case the sensors didn't agree with each other. Yeah. It took months of grounding and a large commercial disaster, but eventually they figured  something could be done after all.
 

Offline TheCalligrapher

  • Regular Contributor
  • *
  • Posts: 151
  • Country: us
Re: C: read() and write().
« Reply #39 on: May 01, 2021, 07:51:18 pm »
I'm not sure what you are trying to say by this. A spurious short read or a short write is not an error at all. It is perfectly normal behavior of `read` and `write`. Why would anyone even think of "exiting" in response to such situation is beyond me.
Because it does not actually happen on modern OSes outside of direct user input (that Ctrl-D stuff) and other extreme scenarios. It is generally not worth handling that.

That's absolutely not true. It appears that you restricted your consideration to a very narrow field of physical files only. Eliminating short reads is only possible when you are working with a regular file in a modern file system: a file system that knows in advance how much data is available in the file and has immediate access to all that data. Yet, POSIX states that even in that case (!) you might end up with a short reads caused by signals.

But `read` offers a much wider functionality than that. `read` can read pipes, `read` can read sockets, `read` can read FIFO files, `read` can read atty-streams and so forth and so on. With all these kinds of inputs short reads is an unavoidable everyday reality. They will always happen. Every time you request 100 bytes from a pipe when only 60 bytes is currently available, `read` will only read 60 and return immediately. `read` will never wait for the other 40 bytes to fullfill your request. `read` is required to do a short read.

This is not an error. This is not an "end-of-data" situation. This is just... normal. Absolutely nothing "extreme" about that scenario.

You just have to remember about it and write your code accordingly. Or, leave `read` alone and just stick to `fread`, which takes care of all these issues for you.

« Last Edit: May 01, 2021, 07:53:35 pm by TheCalligrapher »
 

Online ataradov

  • Super Contributor
  • ***
  • Posts: 11219
  • Country: us
    • Personal site
Re: C: read() and write().
« Reply #40 on: May 01, 2021, 08:12:26 pm »
Yes, I did. Unless it is an explicit requirement for the appliation, I consider only normal files. This may my software inferior and if someone does not want to use it - they are free to do so.

I hate the idea of "everything is a file", so I will not support it in my software.

And sure, if used with sockets, there are a lot of things that need to be handled. But again, this is something that actually happens a lot under normal conditions.
Alex
 

Offline SuntUnMorcov

  • Contributor
  • Posts: 17
  • Country: ro
Re: C: read() and write().
« Reply #41 on: May 05, 2021, 07:19:47 pm »
There is a difference between not checking at all and just "exiting" if an error occurs.

If you just "exit in that situation", that means you have at least checked. Note that some people will even argue that checking for NULL for malloc() is also useless, as, as your argument is, "on modern OSs", dereferencing NULL will trigger an access violation and exit the program anyway. So the motto of this category of developers is: "just let it crash". Seriously.

This sounds all very yucky to me.

Of course if you're mostly writing non-critical tools, that behavior is alright. As I said earlier, it has no consequence, or only very minor.

This is unacceptable for anything critical though. And as I said earlier, when we claim "there is no way we can handle this case anyway", most of the time, it is simply not true. It's often plain laziness mixed with some bad faith. There are many many examples of this of course. When we were talking about the Boeing 737MAX issue, I remember some also claimed there was nothing that could be done in case the sensors didn't agree with each other. Yeah. It took months of grounding and a large commercial disaster, but eventually they figured  something could be done after all.

As someone who works in the aerospace industry, I absolutely second all these points.

Just for some additional insight, I know of some aircraft system failures where the only thing that can be tried (as a last-ditch effort, with *very* slim chances of success) will most likely prove fatal to the passengers of an aircraft.  As these are cases where everyone is going to die anyway, these last-ditch efforts are executed.  But that's all I can say on that subject (due to NDAs - please do not ask me to elaborate).
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 6769
  • Country: va
Re: C: read() and write().
« Reply #42 on: May 05, 2021, 09:58:58 pm »
Quote
There is a difference between not checking at all and just "exiting" if an error occurs.

If you just "exit in that situation", that means you have at least checked. Note that some people will even argue that checking for NULL for malloc() is also useless, as, as your argument is, "on modern OSs", dereferencing NULL will trigger an access violation and exit the program anyway. So the motto of this category of developers is: "just let it crash". Seriously.

This sounds all very yucky to me.

I think you may be swinging right over to one extreme and extrapolating that to everything. Not checking for an error doesn't automatically mean "let it crash". For instance, you may be sending data via UDP and you know that nothing that can be returned affects you - if the data isn't sent... well, it's an unreliable protocol anyway so what do you care? What are you going to do about it? So you just don't bother checking the return - nothing will crash and burn, just that your message doesn't even enter the stack, never mind get lost on the network.

That's very different to not checking the return of malloc() and just using the pointer anyway. But you're classing them both as "just let it crash" mentality.
 

Offline hamster_nzTopic starter

  • Super Contributor
  • ***
  • Posts: 2803
  • Country: nz
Re: C: read() and write().
« Reply #43 on: May 05, 2021, 10:45:47 pm »
Quote
There is a difference between not checking at all and just "exiting" if an error occurs.

If you just "exit in that situation", that means you have at least checked. Note that some people will even argue that checking for NULL for malloc() is also useless, as, as your argument is, "on modern OSs", dereferencing NULL will trigger an access violation and exit the program anyway. So the motto of this category of developers is: "just let it crash". Seriously.

This sounds all very yucky to me.

I think you may be swinging right over to one extreme and extrapolating that to everything. Not checking for an error doesn't automatically mean "let it crash". For instance, you may be sending data via UDP and you know that nothing that can be returned affects you - if the data isn't sent... well, it's an unreliable protocol anyway so what do you care? What are you going to do about it? So you just don't bother checking the return - nothing will crash and burn, just that your message doesn't even enter the stack, never mind get lost on the network.

That's very different to not checking the return of malloc() and just using the pointer anyway. But you're classing them both as "just let it crash" mentality.

The return value of read() and write() seems to fall uncomfortably into the middle ground. Leading to working but somewhat 'snowflake' software. It works well as long as nobody presses CTRL+Z, or you don't redirect to/from a pipe, or a socket or does anything else unexpected.
Gaze not into the abyss, lest you become recognized as an abyss domain expert, and they expect you keep gazing into the damn thing.
 

Online ejeffrey

  • Super Contributor
  • ***
  • Posts: 3669
  • Country: us
Re: C: read() and write().
« Reply #44 on: May 09, 2021, 05:15:49 pm »
There is a difference between not checking at all and just "exiting" if an error occurs.

If you just "exit in that situation", that means you have at least checked. Note that some people will even argue that checking for NULL for malloc() is also useless, as, as your argument is, "on modern OSs", dereferencing NULL will trigger an access violation and exit the program anyway.

The normal argument here was that in a well behaved program (that doesn't allocate obviously wrong blocks of memory) on many systems malloc would normally not fail and just overcommit. Instead if you actually run out if memory you will just thrash to death or get killed by the OOM killer.  This used to be mostly true a long time ago but the overcommit heuristic is a lot better now. 

The argument "don't check for an error my target system will not actually report" is a bit limiting but not intrinsically wrong.

Exceptions are a much more convenient way to handle this.  I dislike exceptions for ordinary handleable results but for an allocation failure where you likely can't recover but might need to do some cleanup it is a nice way to go.
« Last Edit: May 09, 2021, 05:20:16 pm by ejeffrey »
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf