Surprised at Windows Bloat

[@rt]

Hi Guys
I recently installed the current version MS Visual Studio to begin writing a Windows application, which isn’t exactly new to me,
but another couple of programs have been event driven forms with Borland C Biulder that didn’t ask much of the target processor.... and it’s been about a decade.

My first go lately was a timer driven 2D animation, an exact copy of a demo that I’ve run on several platforms, using the first 2D lib I got hold of that worked.
Furthermore, I only use the graphics library to draw pixels of a colour. I do fonts, string printing, circles, lines, etc. all from the ground up on my own.

I was surprised that the Windows demo is so slow. Slower than an original Sony PSP running at 333MHz, slower than a modern iPad, and slower than the same
program running on the same computer under MacOS, and also under iPhone Simulator.

My PC is i5,  4 core, 2.3 GHz that can dual boot Windows 7 or MacOS High Sierra.

Should I be surprised that a Sony PSP running at 333 MHz is after than a multi-core Windows PC running at 2.3GHz?

[ataradov]

It is highly likely that "the first 2D lib I got hold of that worked" is not the most optimal thing for all cases. If it just uses SetPixel() calls, it will be slow, of course.

[@rt]

The line, circle, and text drawing functions all call setpixel() when it boils down to it.
I just stated I did my own drawing to highlight there’s no difference in those functions across all platforms.

[ataradov]

Well, SetPixel() is slow because it is safe and checks that supplied parameters are valid. You can get a raw pointer to the allocated memory and set pixels though a direct memory write. It will be orders of magnitude faster.

[@rt]

Sure there will be plenty of ways to optimise it, but then the program wouldn’t be the same.
My point was that the program is the same across all platforms (at least what I can control), and that the Windows PC is the slowest to run it.

[ataradov]

That API is documented to be slow. It is not designed for animations. I have no idea what assumptions other APIs do, they may not check input parameters.

It is not a windows problem, it is a poorly written universal library problem.

But sure, if you feel like bashing windows for no reason, do continue.

[@rt]

When you say “That API”, what API are you talking about? I haven’t mentioned any API.
I’d be interested to see the documentation for the API you presume I’m using for Windows, especially without my having to tell you what it is.

[Dubbie]

That's why games on windows run 10-20x slower than on other platforms....

Oh wait, no they don't.

Whats that old saying about a poor workman blaming his tools?

[@rt]

Hang on.. I didn’t complain about speed. Only the relative difference between platforms  -> For exactly the same program <-

How does that make me a poor workman?

[ataradov]

When you say “That API”, what API are you talking about? I haven’t mentioned any API.
I’d be interested to see the documentation for the API you presume I’m using for Windows, especially without my having to tell you what it is.

By SetPixel() I actually meant Win32 API function with that name. If it is not the case, then how exactly that library does the actual drawing?

Games use DirectX and OpenGL, and drawing there pixel by pixel will be even slower.

[ataradov]

Hang on.. I didn’t complain about speed. Only the relative difference between platforms  -> For exactly the same program <-

It is not exactly the same if it touches platform-specific APIs to draw pixels on the screen. Or can you exclude drawing from the performance measurement?

[@rt]

It wouldn’t be reasonable to expect me to have looked much at the graphics lib (particularly it's set pixel that I’m using),
as MS Visual Studio has literally only been installed for a few days, and I wanted a demo running to se that everything worked.

Excluding the drawing is easy, and I suspect that it will still be slower. I’ll be back.

and @Dubbie, Someone else’s graphics library and Windows has nothing to do with my code. That’s someone else’s workmanship,
even if my code is the worst in the world, that’s not at all relevant.

[Nominal Animal]

The line, circle, and text drawing functions all call setpixel() when it boils down to it.

No. Or more correctly, the component that does the equivalent operation varies. You assume it is always the main CPU, and that is not correct.

The way graphics operations can be done so fast today (just look at e.g. Blender 2.8 and its EEVEE realtime renderer) is that the main CPU does not do them at all. The code that does the closest thing to a "setpixel()" call executes in the GPU, often on a separate graphics card. The CPU only lists operations and objects (like textures or images, vertices for polygons, and so on), and the very specialized GPU does all the work.

It definitely matters what library or approach you are using. If you insist on doing every setpixel() call on a CPU in your own code, you should expect it to be slow, because that's not how the entire system is designed to operate efficiently.

That said, I use Linux myself, and don't touch Windows at all. The same thing applies, except that Linux also supports non-accelerated display devices with framebuffer support (i.e., display drivers without a GPU to do operations, or perhaps just a few 2D operations). You can run raw (GNU/POSIX) C code to drive them, or use existing libraries to do most of the hard work. Some of them use advanced features of various CPUs to do the job faster than others. Some of them work well on certain hardware, worse on others.

[@rt]

That sounds a lot like the Sony PSP, where I have also insisted on calling setpixel() for line, circle, and font/string functions.

[rjp]

setpixel on windows is wrapped  in a memory region lock and unlock and is notoriously slow.

variations on https://stackoverflow.com/a/24702215 are the correct approach

[@rt]

Yeah ok... I removed calls to set pixel(0, and it’s much faster. It can certainly be narrowed to drawing.

@rjp, at a glance, it looks like it would be better to write to a bitmap in my own memory and write the bitmap to the window instead.
This would be especially easy since I’m planning a fixed size window.

[ataradov]

@rjp, at a glance, it looks like it would be better to write to a bitmap in my own memory and write the bitmap to the window instead.

You will still have to go through the same process. You will need to prepare everything in a format that BitBlt() understands, and it will be HDC. And the only straightforward way to draw into HDC is the same WinAPI. So it does not really matter which HDC you get raw access to. So the StackOverflow advice still applies.

But drawing things into a backing bitmap is a good idea to avoid tearing.

[Mr. Scram]

Did you attempt to write a Windows Forms application? Those really aren't intended for speedy 2D animations.

[@rt]

It’s already double buffered by using two pages, as otherwise the screen flickered.
I could remove that I suppose if I was using a bitmap, and end up with he same thing.
I’m on a phone now, but will better read the link later.

@ Mr Scram, no I wrote a couple of event driven form programs years ago.

[rjp]

Yeah ok... I removed calls to set pixel(0, and it’s much faster. It can certainly be narrowed to drawing.

@rjp, at a glance, it looks like it would be better to write to a bitmap in my own memory and write the bitmap to the window instead.
This would be especially easy since I’m planning a fixed size window.

this shows the basci pattern for getting at the bitmaps pixels directly - just focus on the "destination" image api  to see how you can manipulate the in memory byte array with zero overhead.

a fast copy from a byte array you prepared yourself is shown at this link

https://msdn.microsoft.com/tr-tr/library/system.drawing.imaging.bitmapdata%28v=vs.110%29.aspx

[IanB]

Hi Guys
I recently installed the current version MS Visual Studio to begin writing a Windows application, which isn’t exactly new to me,
but another couple of programs have been event driven forms with Borland C Biulder that didn’t ask much of the target processor.... and it’s been about a decade.

My first go lately was a timer driven 2D animation, an exact copy of a demo that I’ve run on several platforms, using the first 2D lib I got hold of that worked.
Furthermore, I only use the graphics library to draw pixels of a colour. I do fonts, string printing, circles, lines, etc. all from the ground up on my own.

I was surprised that the Windows demo is so slow. Slower than an original Sony PSP running at 333MHz, slower than a modern iPad, and slower than the same
program running on the same computer under MacOS, and also under iPhone Simulator.

My PC is i5,  4 core, 2.3 GHz that can dual boot Windows 7 or MacOS High Sierra.

Should I be surprised that a Sony PSP running at 333 MHz is after than a multi-core Windows PC running at 2.3GHz?

There are a multitude of ways to write graphics programs using Visual Studio. What language environment and graphics library are you using? (For example native Windows API a la Petzold with C, or MFC with C++, or Windows Forms with .NET, or WPF, or...?)

[timgiles]

You seem to be suggesting by your question that the only difference between these environments that you are running code on, is the software environment, or the speed of the GPU.

The API in question does not write directly to the hardware and depending on the API there can be quite a few layers of abstraction and drivers to get through before the work is done. Also, some of the platforms you are referencing will have hardware and drivers that support certain operations much better than others. 2D on the PC platform stagnated in the mid 2000s - I remember reading an article I think on Toms Hardware that was trying to point out graphics card manufacturers that the 2D performance of pushing pixels, drawing fonts etc.. had neither improved and in some cases gotten worse, whilst, screen size/resolution had increased.

Windows has improved significantly since 7. The Windows Display Driver Model back in the days of 95 and 7, a program could easily cause the whole screen to stop updating and either with windows 8 (WDDM 1.2) that was finally fixed to allow rendering of the display in separate threads. And with Windows 10, we have version 2.0. It was a long time coming and it took a while for the main GPU vendors to get their drivers fully updated - but still a marked improvement.

[@rt]

There are a multitude of ways to write graphics programs using Visual Studio. What language environment and graphics library are you using? (For example native Windows API a la Petzold with C, or MFC with C++, or Windows Forms with .NET, or WPF, or...?)

It’s an old thing from about 2004 that I think is called Windows BGI. It came with a demo which was a pair of eyes that followed the mouse round the screen.
That was event driven by mouse movement, but I removed that and stole a timer from another library.
With .cxx extensions, I assume that is still C++ (I write in C). I’m not in love with it, and open to suggestions, but being so new with Windows,
anything new would have to come with a complete project demo that at least draws a pixel, and is preferably double buffered.

[technix]

Since you are talking about animation, it might be a better idea to go with DirectX or OpenGL, and let the GPU do all the drawing and SetPixel()’ing. Or you can just grab a game engine like Amazon Lumberyard, which is a fork of Crytek CryEngine.

If the animation is fixed, it might be easier to create it using some video animation software like After Effects or Blender and pre-render it into a video file, and play it back when needed.

Do be aware that Lumberyard can be very taxing on the GPU, since the Crysis series of games developed on CryEngine has a reputation of being very demanding.

[Nominal Animal]

For what its worth, I've found that Qt + Python yields quite good 2D performance, when using QPainter and its drawThing() methods. Qt is also an obvious choice for cross-platform C++ applications, although in some cases FLTK or wxWidgets might be preferable. For C, I really like GTK+. It too is cross-platform (and uses Cairo).

For games, there's SDL. For just 2D graphics, there's Cairo.

Those are the libraries I'd look at, and pick the one that suits my needs best. They all provide accelerated primitives (meaning the GPU does most of the work on all systems that have a GPU). wxWidgets uses the OS/Desktop Environment widget toolkit to achieve a native look and feel, the others have their own themes (and theming support); Qt's default theme emulates the native look-and-feel on different systems. SDL is obviously more realtime/gaming oriented. You can use Cairo to generate images even without any graphical user interface being available, i.e. for command-line applications too.

[@rt]

I don’t think I’ll ever be interested in anything that isn’t mine from the ground up.
Even for Apple iOS Apps, it was the same. Never once used any of their forms, or whatever you call them, it was all my own 2D.

There’s still a few bits of hardware I’m waiting for to get right into the Windows thing.
The animation was just a test that things worked, but I’ll certainly refer to the links here.

It’s been on my mind to rid myself of all Apple products for some time. Export all iTunes music as MP3, etc.
The very last reason to hold onto my Mac was to run Xcode for iOS Apps. I had five Apps in the store some years ago.
They were booted from devices when iOS started requiring 64 bit Apps, but always ran on my Mac because I never updated
Xcode, the SDK, or anything else, but stopped working on my (up to date) target devices of course.

I was sure an Xcode update would break everything, and the first build did report some 350 depreciated function calls,
but low and behold it’s running on the latest iOS version on a fairly new iPhone. I never thought I’d see that again.
Now there’s an educational institution willing to pay my developer subscription, so I’ll get them back into the store!

[Nominal Animal]

I don’t think I’ll ever be interested in anything that isn’t mine from the ground up.

Well, it's a matter of where you draw the line, really. I don't suppose you've written your own OS? 

Me, I like Linux, because I can make it my bitch do what I want; to conform to my needs, and not vice versa. I also like the modular approach as described in the Unix Philosophy, "Do one thing and do it well"; for most tasks, I combine several tools rather than try to solve it all in one application. I don't like frameworks (and you could say Qt, GTK+/GDK, FLTK, wxWidgets, and SDL are application frameworks), so I tend to keep my processing logic quite separate from the UI -- unless the processing logic is trivial.

Because I know I can fiddle (and have fiddled) with all parts of the software stack, from the kernel to C libraries to the various services up to widget toolkits, I don't mind choosing my "line" on case-by-case basis.  I do prefer to write my own implementations of the critical parts, because I've seen (and occasionally fixed) so many bugs that I really think most programmers should not be allowed to distribute their code.

[@rt]

I don’t think I’ll ever be interested in anything that isn’t mine from the ground up.

Well, it's a matter of where you draw the line, really. I don't suppose you've written your own OS? 

I have, but only where the hardware was mine as well, so there was no OS to begin with.

I’m also sorry to say that iOS was my favourite platform to write for, but only after abstracting access to their hardware in a manner allowing me to eventually just write in plain C, and never look at Objectionable C again.
I’m not going back there though.

[Mr. Scram]

Using libraries often means not spending weeks optimising the same thing others have already optimised. Reinventing the wheel is a common pitfall. As long as you make sure that what's you're using isn't actually crap you can save yourself quite a bit of trouble.

[Dubbie]

Some "simple" things like a database, graphics library or even a KDTree can take YEARS of work to get perfect.

I don't know why you would waste time redoing those sorts of things. Your solution is bound to be worse in almost every way.

Surely you'd rather spend the effort on the parts you are writing that are actually new?

Unless this is just a way to keep yourself busy indefinitely and never finish anything. In that case, carry on!

[@rt]

Some "simple" things like a database, graphics library or even a KDTree can take YEARS of work to get perfect.

I don't know why you would waste time redoing those sorts of things. Your solution is bound to be worse in almost every way.

Surely you'd rather spend the effort on the parts you are writing that are actually new?

Unless this is just a way to keep yourself busy indefinitely and never finish anything. In that case, carry on!

Moving the demo project from one platform to the other literally took 15 minutes,
and doing anything else would have provided an even less fair comparison between the two.

[Mr. Scram]

Moving the demo project from one platform to the other literally took 15 minutes,
and doing anything else would have provided an even less fair comparison between the two.

What exactly are you trying to demo? Because without leveraging the strong suits of a platform and taking its weaknesses into account it's not a very valuable comparison in the first place. Drawing 2D images pixel for pixel with an unoptimized homebrew engine is going to have predictable results without telling you much about what it could be.

[tooki]

I don’t think I’ll ever be interested in anything that isn’t mine from the ground up.
Even for Apple iOS Apps, it was the same. Never once used any of their forms, or whatever you call them, it was all my own 2D.

That's a terrible approach, though, for so many reasons. As others have said, libraries (especially first-party) have been extensively optimized over the years. But other huge reasons to use existing libraries/APIs include accessibility (the OS's own libraries usually give you accessibility support for free) and UI consistency (it's hugely annoying when apps use custom widgets that don't quite behave like the standard ones). So in the end, you're wasting time — time you could have spent on app features, not reinventing the wheel of the internal plumbing — to produce something that's almost guaranteed to be slower, buggier, have less functionality, and have lower usability. It's a lose-lose situation.

I worked at a software company where the lead dev decided it'd be "easier" to base all the controls off of a highly customized Win Forms text box. Jeebus H Crikes, it caused so many problems. High DPI? Broken. Screen reader? Broken. Some language versions of Windows? Broken. Undo? Broken.



Ultimately — and I say this as a general rule, not as an insult to you — I think that devs that do this frequently suffer from a sort of Dunning-Kruger effect. They think they're above-average programmers (not recognizing that the folks who write OS libraries/APIs are insanely smart) and can do it better, but in fact, they don't know what "good" is, and are actually below average. So they reinvent the wheel, but instead of round, it's 7-sided with an eccentric shaft…

[technix]

I don’t think I’ll ever be interested in anything that isn’t mine from the ground up.
Even for Apple iOS Apps, it was the same. Never once used any of their forms, or whatever you call them, it was all my own 2D.

There’s still a few bits of hardware I’m waiting for to get right into the Windows thing.
The animation was just a test that things worked, but I’ll certainly refer to the links here.

It’s been on my mind to rid myself of all Apple products for some time. Export all iTunes music as MP3, etc.
The very last reason to hold onto my Mac was to run Xcode for iOS Apps. I had five Apps in the store some years ago.
They were booted from devices when iOS started requiring 64 bit Apps, but always ran on my Mac because I never updated
Xcode, the SDK, or anything else, but stopped working on my (up to date) target devices of course.

I was sure an Xcode update would break everything, and the first build did report some 350 depreciated function calls,
but low and behold it’s running on the latest iOS version on a fairly new iPhone. I never thought I’d see that again.
Now there’s an educational institution willing to pay my developer subscription, so I’ll get them back into the store!

For Apple recent years was a huge paradigm and mode shift for them - it is now iOS leading the kernel development, Swift is now front and center instead of Objective-C, etc. No wonder low level stuff break - they just plain wont fit in the newer environment, and the glue just won’t be efficient enough. If you programmed your apps using high level libraries Apple has made it almost one recompile away from supporting 64-bit iOS devices.

Apple’s music is encoded in AAC, widely supported in most recent devices since it is so widely used.

[Nominal Animal]

folks who write OS libraries/APIs are insanely smart

No, they're not, unfortunately.  (I'm approximating "insanely smart" ≃ "one in ten thousand", so that there is approximately 750 000 of them on Earth right now.)

Some of them are experienced and create very robust, very efficient code, and others are just productive.  Very few are insanely smart.  Based on the code they produce, I'd say djb is one of the rare insanely smart ones; a lot of people dismiss him and his work right out of hand.  Most of the programmers you see in the author lists in OS development, library and compiler development, and even huge software projects, are merely very productive. Some of their code is utter crap, and a lot of their designs are fragile or inefficient.

Productive programmers concentrate on the new and interesting features, and expect others (typically system integrators and end users) to debug their code.  Yes, they're innovative and productive; but not necessarily very smart; definitely not insanely smart.  Some of them (look at e.g. systemd) even use fanboi fawning as a defence against having to fix the bugs they themselves caused, so I consider elevating developers who do not objectively deserve such, as actively harmful.

Insanely smart developers tend to produce not that much code. They seem to find a way to simplify most problems completely away; they do a lot with surprisingly little.

Some large projects like GCC and the Linux kernel are amazing, but I would not call their developers insanely smart either.  They're very, very knowledgeable. You don't need to be insanely smart to create an amazing software project, but you do need to be industrious (since a lot of development is drudge work, taking care of details), innovative (in finding ways to solve many problems with not much effort), and productive.

[@rt]

I grew up with old computers and microcontrollers, and that’s what I think the influence is.
I’d say I’m creative, persistent and focussed, but not insanely smart.

Nothing has really been written for Windows by me. I only moved a 2D demo over to MSVS, and of course,
without knowing the API, it was easy to move all 2D graphics, and just replace setpixel();, figure out how to double buffer, figure out a hardware timer, etc.
This was basically to test things work, and then I started the thread being very surprised at the comparison.
This is the actual demo on Windows 7 with i5 CPU:

It’s a fixed 480x272 window because it came most directly from Sony PSP code where it runs 60fps.
I did comment the drawing, and it is the drawing BTW. The point rotation, etc will still run fast on it’s own invisibly.

Now for iOS Quartz 2D, there was actually no set pixel function. The best you could do is draw a single unit length line,
which would require more arguments sent through the stack than a proper setpixel();.
That’s where I can grab some of my existing code to easily compare the overall technique of sending a bitmap to Windows.
None of this is the result of “I can do it better”, just “I can do it”.

[technix]

I grew up with old computers and microcontrollers, and that’s what I think the influence is.
I’d say I’m creative, persistent and focussed, but not insanely smart.

Nothing has really been written for Windows by me. I only moved a 2D demo over to MSVS, and of course,
without knowing the API, it was easy to move all 2D graphics, and just replace setpixel();, figure out how to double buffer, figure out a hardware timer, etc.
This was basically to test things work, and then I started the thread being very surprised at the comparison.
This is the actual demo on Windows 7 with i5 CPU:

It’s a fixed 480x272 window because it came most directly from Sony PSP code where it runs 60fps.
I did comment the drawing, and it is the drawing BTW. The point rotation, etc will still run fast on it’s own invisibly.

Now for iOS Quartz 2D, there was actually no set pixel function. The best you could do is draw a single unit length line,
which would require more arguments sent through the stack than a proper setpixel();.
That’s where I can grab some of my existing code to easily compare the overall technique of sending a bitmap to Windows.
None of this is the result of “I can do it better”, just “I can do it”.

Throw this thing on a Dell P2415Q 24-inch 4K monitor it would be unreadable.

[@rt]

Well I’ve done the same thing for iPhone/iPad, PSP, and even the PSION 5mx PDA now, in their native screen resolutions, so it’s kind of become a benchmark thing.

The actual program I have in mind to write, there’s no escaping assembling a bitmap for each frame, and firing off graphics frames with a timer.
If someone else wrote it, it would essentially be the same thing, but live in a form I suppose, with Windows buttons, but the dynamic 2D would still have to be there.

[@rt]

year.. ok.. I just had to set pixels in a bitmap.
It’s 24 bit, but I haven’t bothered setting colour components yet.
Thanks for replies (especially links).

[Tony_G]

I read the thread but I can't remember if you were pointed to Direct2D (it was mentioned that SetPixel is about the slowest way to achieve your goals):

https://docs.microsoft.com/en-us/windows/desktop/Direct2D/direct2d-overview

The background bitmap approach you've taken will probably get you the best minimal change cross platform code but if you move over to D2D then you'll be able to do much more with your visualizations and even things like rich text experiences and localization through DirectWrite.

However your code will become way less cross platform and be more complex, though less complex than trying to implement these things yourself.

If you really want to go the whole hog cross platform it might be worthwhile to explore OpenGL.

TonyG

[tooki]

FYI, Apple has officially changed its OpenGL support to “deprecated”. It’ll still work for another version or two of macOS and iOS, but it’s on the chopping block. They really want things in Metal (just as MS wants everything on DirectX). So OpenGL is probably not a forward-looking cross-platform solution.