Python becomes the most popular language

[Picuino]

runs at full speed (although it does not optimize the generated code)

That is as useful as a recursive-only Fibonacci calculator.  It, too, runs at full speed; it's just that the results aren't that useful in practice, because there are much better options around.

I find it useful to have a C interpreter that can quickly execute a script when what matters (again) is the speed of the programmer.

Once the program is developed, you can compile it with GCC to have the executable already compiled and faster. But during development, a compiler that compiles the whole project 10 to 15 times faster and runs in memory (without producing an executable file) can be an advantage.

For example to develop code that will later be ported to a microcontroller. The development phase will be much faster with a C interpreter like TCC.

I only intend to highlight the advantages of interpreted code over compiled code. And I don't care if others don't find advantages in Python. I do easily find the advantages of Python and that's why I use it where I think it will do well, which is not to build castles.
In other cases I have used C or JavaScript, where Python didn't make sense.

[Picuino]

I am aware that there is big difference in getting compiled and interpreted stuff going, but surely that isn't what makes a language. You can get C interpreters and you can compile Python applications if you really want.

Unfortunately, Python code cannot be compiled. I wish it could be done, even at the cost of removing capabilities from the language. But today it can only be interpreted.

[SiliconWizard]

One thing that saddens me is that many (I didn't say all) people who use Python almost refuse to consider alternatives. That's pretty sad (and unreasonable.)
"Monopoles" are never a good thing and they never end up well.

[Picuino]

That's because Python is a religion 

Seriously, I don't think there are any Python-only programmers. There may be C-only programmers, but Python-only programmers, impossible.

[Nominal Animal]

Hmmm. I think my idea was either not expressed usefully or it failed to be noticed (which probably comes to the same).

The latter is always possible with me, because me fail English often; I miss subtext and cues.

I am aware that there is big difference in getting compiled and interpreted stuff going, but surely that isn't what makes a language. [...] So, are we really saying that the language's lexical features aren't actually very important, and what everyone is liking is the ability to type into a live system and watch run or crash as they're doing that?

No, we (I believe!) are saying that a small detail in one context can be a big factor in another context.  That is, that there isn't one set of weights that would apply generally, that one could use to calculate a single qualitative scalar for describing the usefulness of a programming language.

Or, put another way, that the uses of programming languages vary so wildly, that there physically cannot be a single programming language that fits them all.

A better question would be, when does the interpretability/scriptability of a language override its lexical/library features, or vice versa?

(I have my own answer, already outlined in my posts where I describe how and when I use Python and why, but it definitely isn't "universal".  It would be interesting to me to hear when others find themselves crossing that line.  I expect use cases where domain-specific languages were traditionally used, would be quite informative.)

As to C standard library, I find it quite deficient, and instead prefer POSIX C (which basically all operating systems' standard C libraries implement, except for Windows).  I've also started a couple of threads here talking about replacements for the standard C library, as it is not an intrinsic part of the language: the standard specify the freestanding environment, where the standard library is not available.  Indeed, freestanding C/C++ is, I believe, the most common choice for software development on microcontrollers.

I find it useful to have a C interpreter that can quickly execute a script when what matters (again) is the speed of the programmer.

Okay, that would be an valid use case.  I stand corrected.

The reason I didn't think of that, is that the way I develop my own projects, means that I don't actually ever need to wait for the compiler.  Key algorithms and verification suites –– like the Xorshift64* with Marsaglia polar method PRNG I wrote yesterday elsewhere, to generate normally distributed pseudorandom floats on Cortex-M7 –– I first implement in separate programs to test and verify their operation, before including in the larger project.  For larger projects (and even smaller ones), I use Make extensively, so only the recently modified sources are recompiled and then the project linked, and even on this five-year-old HP EliteBook 840 G4 laptop with an Intel Core i5-7200U processor, I never need to wait.  I would not save any time even if the compiler was faster.

For larger projects, like the Linux kernel, I run the build on the background (nice'd and ionice'd) while I check the Changelog, documentation, et cetera.  Even there, the compilation itself isn't necessarily the slowest part of the build; there are lots of dependency tracking and such, which is why you'll see massive improvement in build speed if you have more RAM (because Linux uses otherwise unused RAM as a cache, which speeds up random access times).  Of course, now with fast SSD drives, the difference has diminished quite a bit –– but not completely, because Linux is one of those projects with LOTS of small files accessed during the build, so it's more about I/O requests per second than about raw data bandwidth.  Caching still helps.

I only intend to highlight the advantages of interpreted code over compiled code.

I've described how I do that too.  I'm not trying to counter anyone here; I too am just trying to help concentrate on the useful and practical aspects of this discussion, and not let it devolve into an opinion shootout.

[ve7xen]

Unfortunately, Python code cannot be compiled. I wish it could be done, even at the cost of removing capabilities from the language. But today it can only be interpreted.

With Cython + any old C compiler you can do exactly that.

The difference between 'compiled' and 'interpreted' is generally more about the toolchain/implementation than the language itself. It's hard to imagine a language that could be interpreted but not compiled, unless you write implementation details into the language spec. The lines get very blurry when you're talking about tools/runtimes that use an intermediate bytecode (like CPython) or a JIT (like PyPy). Java is traditionally considered a 'compiled' language, but its has an intermediate representation not unlike CPython's - it doesn't compile directly to native code - the tools just require you to compile it ahead of time with Java but do it at runtime (and cache it) in CPython. In fact, there are Python implementations that compile to other intermediate languages and use their runtimes, like Jython (Java) or IronPython (.NET CLR).

One thing that saddens me is that many (I didn't say all) people who use Python almost refuse to consider alternatives. That's pretty sad (and unreasonable.)
"Monopoles" are never a good thing and they never end up well.

I think this is as true of Python as it is of any other language. Perhaps Python's flexibility lends to this being a more viable position than with many other languages, but it is human nature to continue using a tool you know rather than invest the effort into one that might or might not suit you better.

Seriously, I don't think there are any Python-only programmers. There may be C-only programmers, but Python-only programmers, impossible.

Eh, I'd actually say the opposite is more likely. Like there are embedded programmers that started with Arduino and never moved past it, the same will be true of Python. It's flexible and 'good enough' that it can solve pretty much any normal programming problem in a reasonable way, so why learn something else? Whereas if you know C, you probably invested a lot of time into it, and along the way likely learned at least one other language.

[Nominal Animal]

One thing that saddens me is that many (I didn't say all) people who use Python almost refuse to consider alternatives. That's pretty sad (and unreasonable.)
"Monopoles" are never a good thing and they never end up well.

Isn't that universal, though?  I mean, we are all surely familiar with the single-tool-fits-all-my-needs people, in all aspects of life.  Python is no exception and no different to any other programming language, or even any other tool, really.

The fact that this is so common, does make me sad too, though.  The older I get, the lesser my overall opinion of humanity becomes, and the more I value individuals who try to make a difference around themselves, in the small things, by helping others.

[PlainName]

I am aware that there is big difference in getting compiled and interpreted stuff going, but surely that isn't what makes a language. You can get C interpreters and you can compile Python applications if you really want.

Unfortunately, Python code cannot be compiled. I wish it could be done, even at the cost of removing capabilities from the language. But today it can only be interpreted.

py_compile?

But what I mean but 'compile' is really 'single standalone executable', and there are utils that will do that.

[nctnico]

I am aware that there is big difference in getting compiled and interpreted stuff going, but surely that isn't what makes a language. You can get C interpreters and you can compile Python applications if you really want.

Unfortunately, Python code cannot be compiled. I wish it could be done, even at the cost of removing capabilities from the language. But today it can only be interpreted.

py_compile?

But what I mean but 'compile' is really 'single standalone executable', and there are utils that will do that.

Only to a certain extend. You'll still need to figure out a way to pull all the system libraries in that the Python modules depend upon. Deploying a single binary that depends on a few libraries so it runs on various OS versions/flavours isn't easy. Deploying a lot of binaries that depends on even more libraries is very hard.

[PlainName]

That's not far different from getting a compiled app to run somewhere else. You either include everything needed or have the user install 'common libraries' which, in the case of an interpreter, is the full stack.

[pcprogrammer]

I am aware that there is big difference in getting compiled and interpreted stuff going, but surely that isn't what makes a language. You can get C interpreters and you can compile Python applications if you really want.

Unfortunately, Python code cannot be compiled. I wish it could be done, even at the cost of removing capabilities from the language. But today it can only be interpreted.

py_compile?

But what I mean but 'compile' is really 'single standalone executable', and there are utils that will do that.

Only to a certain extend. You'll still need to figure out a way to pull all the system libraries in that the Python modules depend upon. Deploying a single binary that depends on a few libraries so it runs on various OS versions/flavours isn't easy. Deploying a lot of binaries that depends on even more libraries is very hard.

Not to run on different operating systems but for linux this exists. https://appimage.org/

[Picuino]

What I mean when I say that Python cannot be compiled is that it cannot be converted into a static machine language. Although a Python application can be packaged into a single executable, it will contain the Python interpreter that will be dedicated to interpreting the intermediate object code generated earlier.
The final appearance is that of a compiled executable, but the interpretation is still done. Proof of this is that you can internally generate new Python code or enter new code in a keyboard entry and run it with the "compiled" application, which will be interpreting the new code.

Edit:
One consequence of this is that "compiled" Python code is not much faster. Variables can keep changing type during program execution, so the program has to check the type of each variable before using it. It is still just as slow as interpreted code.

[brucehoult]

What I mean when I say that Python cannot be compiled is that it cannot be converted into a static machine language. Although a Python application can be packaged into a single executable, it will contain the Python interpreter that will be dedicated to interpreting the intermediate object code generated earlier.
The final appearance is that of a compiled executable, but the interpretation is still done. Proof of this is that you can internally generate new Python code or enter new code in a keyboard entry and run it with the "compiled" application, which will be interpreting the new code.

Edit:
One consequence of this is that "compiled" Python code is not much faster. Variables can keep changing type during program execution, so the program has to check the type of each variable before using it. It is still just as slow as interpreted code.

While all that you say is true, JavaScript has exactly the same characteristics, and yet using JIT techniques it runs faster than unoptimised C and very close to optimised C.

It does of course have to carry the interpreter/JIT along with it.

[Picuino]

It is true that Python could have a better performance in execution speed and that has been remarked many times here. In fact, improving its speed is a topic that is being worked on today.

But what I have commented on several occasions and I do not know if it has been well understood is that Python has a lot of performance from the programmer's point of view because:
1. you can do things with Python that you wouldn't know how to do or want to do with other languages.
2. With Python it takes little effort to make programs that would not be feasible in C or other languages because of their complexity.
3. Python is more maintainable than other languages because it helps create code that is easy to read. In the long run we spend more time maintaining and improving existing programs than programming new ones.

Therefore Python is a very good choice in cases where what matters is programming time, not execution time (which is becoming less and less important in many areas).

[madires]

Execution time and power efficiency are becoming more and more important because of the power consumption and CO2 footprint. Might change some day when we'll have fusion power plants, but not for the foreseeable future.

[Picuino]

You are right. So am I.
No matter how much of a carbon footprint it has, don't expect websites written in PHP to be rewritten in C.
Wordpress (written in PHP) dominates CMS. But it's not the only one out there. SMF and phpBB in forums, Matomo in Analytics, etc.

Alternatives? Perhaps Python, Ruby, JavaScript, all of them interpreted, not compiled and therefore with a larger carbon footprint. That's the way it is.

[Picuino]

I once read that Python had failed to be used for Android apps precisely because it was slower than Java. In smartphones, saving energy is important, but anyway a compiled language (Java), not C, is used to program the Apps.

https://stackoverflow.com/questions/3560963/why-does-android-use-java

[nctnico]

I am aware that there is big difference in getting compiled and interpreted stuff going, but surely that isn't what makes a language. You can get C interpreters and you can compile Python applications if you really want.

Unfortunately, Python code cannot be compiled. I wish it could be done, even at the cost of removing capabilities from the language. But today it can only be interpreted.

py_compile?

But what I mean but 'compile' is really 'single standalone executable', and there are utils that will do that.

Only to a certain extend. You'll still need to figure out a way to pull all the system libraries in that the Python modules depend upon. Deploying a single binary that depends on a few libraries so it runs on various OS versions/flavours isn't easy. Deploying a lot of binaries that depends on even more libraries is very hard.

Not to run on different operating systems but for linux this exists. https://appimage.org/

I remain sceptical until I see it actually work  but this looks like an interesting solution I'm going to look into when I revisit the Python project that has hit a dead-end due to lack of distributability. The alternative is rewriting it in C++.

[pcprogrammer]

I remain sceptical until I see it actually work  but this looks like an interesting solution I'm going to look into when I revisit the Python project that has hit a dead-end due to lack of distributability. The alternative is rewriting it in C++.

Don't think it is written in python but saleae use it for their software. Works like a charm. Just the one file that you can execute. https://www.saleae.com/downloads/

[eti]

I got a message for wannabe programmers:

When children who wanna become mathematicians grow up, they leave behind their 10 times table and work on REALLY, REALLY difficult and complex problems.

Programming IS hard, because you're solving very difficult problems. You want the (imho mis-attributed) title of "software engineer", but ya don't want to put in the hard graft of learning what is ACTUALLY underneath all those layers? When you learn the low down stuff, you become a better programmer as you mentally visualise what's happening at the bare metal. I'm not saying you have to learn ASM, I mean it wouldn't hurt you, but at the LEAST become semi-proficient at C, even if you don't use it in daily work. Write some code then examine the ASM the C compiler spits out for you; you'll learn about stacks and pointers and registers. 

Real engineers learn ohm's law, how transistors work etc, they don't plug together arduinos and say "Ehhhh, good enough; I'm an engineer now" - software is no different. Python is a toy, you can learn it once you've learnt the hard stuff.

People want all the praise without the work. Do something EXTREMELY HARD and work it out for yourself, you'll have a fundamental understanding of the bits and bytes and then you can build on that foundation. Drag and Drop IS NOT programming.

[Picuino]

Pyratemp: http://www.simple-is-better.org/template/pyratemp.html
Pyratemp is a very simple, easy to use, small, fast, powerful, modular, extensible, well documented and pythonic template-engine for Python.


Simple? only a Toy? 
1230 lines of code. One complete template languaje interpreter.

[eti]

When I need to run certain things in Ubuntu, which depend on Python, I almost ALWAYS come across daft problems like needing 2.7 vs 3.X, having to search stack overflow etc for myriad, conflicting "solutions", and there being multiple conflicts etc. It's not the best first impression. 

[SiliconWizard]

One thing that saddens me is that many (I didn't say all) people who use Python almost refuse to consider alternatives. That's pretty sad (and unreasonable.)
"Monopoles" are never a good thing and they never end up well.

Isn't that universal, though?  I mean, we are all surely familiar with the single-tool-fits-all-my-needs people, in all aspects of life.  Python is no exception and no different to any other programming language, or even any other tool, really.

Not really. It is universal for anything that has been considered a 'silver bullet', and those never last. They are just passing fads.

Just look at C. While it's still hugely popular if you consider its age, it is now restricted to niches (embedded software, maintenance of existing Linux/BSD/.. software, Linux kernel...) No one is currently trying to push it to be used universally for all purposes. Same for all other currently used programming languages. If that was once the case, the fad has passed.

So the same will happen with Python. But currently, its proponents are eagerly trying to push it to almost anything conceivable, going out of their way to try and find new applications for it at all costs. That's not sane and it will definitely not last - it never has in the past. If it does last, then we are in a seriously troubled period IMO. (Which uh, we are indeed, though.)

The fact that this is so common, does make me sad too, though.  The older I get, the lesser my overall opinion of humanity becomes, and the more I value individuals who try to make a difference around themselves, in the small things, by helping others.

Yeah. And to restrict things to the thread, I also try at my humble level to use (and show to those potentially interested) a variety of tools and languages that are sometimes underrated and worth a look. This promotes some variety and healthy "competition" and expands your horizons. I again don't think that obsessively focusing on just one thing is sane, and that currently troubles me with Python, but of course not just that. Purely my opinion here.

[Picuino]

It seems that the first 4 languages are taking the upper hand and leaving less and less room for the others.

"""
TIOBE Index for October 2022
October Headline: The big 4 languages keep increasing their dominance
Python, Java, C, and C++ are the top 4 languages of the TIOBE index for quite some time now. They are far ahead of the rest, and the gap seems to be only growing. Last year, these 4 languages had a 40% market share, this year the share is even 55%. This is an indication that there is not much room for competition at the moment. Java's main competitor C# is losing ground, whereas Python competitors R and Ruby are more or less stabilizing. However, for C and C++ there is one small threat arising on the horizon: the programming language Rust. Rust re-entered the top 20 again with an all time high market share of 0.70%. Rust focuses on performance and type safety. One of the reasons for Rust's increasing popularity is that Google started programming low level parts of Android in the Rust language. -- Paul Jansen CEO TIOBE Software
"""
https://www.tiobe.com/tiobe-index/

(Of course, speaking of popularity, which may not mean anything).

[Picuino]

It seems that they are working on solving one of the problems that "worry" so much about Python, its slow speed:

https://docs.python.org/3.11/whatsnew/3.11.html#whatsnew311-faster-cpython
"""
Faster CPython

CPython 3.11 is on average 25% faster than CPython 3.10 when measured with the pyperformance benchmark suite, and compiled with GCC on Ubuntu Linux. Depending on your workload, the speedup could be up to 10-60% faster.

This project focuses on two major areas in Python: faster startup and faster runtime. Other optimizations not under this project are listed in Optimizations.
"""