EEVblog Electronics Community Forum

Products => Computers => Programming => Topic started by: Sherlock Holmes on November 22, 2022, 05:49:21 pm

Title: The Imperium programming language - IPL
Post by: Sherlock Holmes on November 22, 2022, 05:49:21 pm
I've been pondering the idea of attempting to create a new (compiled) programming language specifically designed for hardware programming, microcontrollers. The impetus for this is primarily a frustration and disappointment with the popular languages that are out there like C++, Rust and C.

This is not as crazy as it might at first appear, I'm a very experienced software developer and very experienced with C as well as other languages on mainframes, minicomputers and Windows. I have also implemented a full working compiler before for Windows NT including code generation, optimization and COFF object file creation, I'm also trained in electronics and telecommunications but never worked professionally in that field, entering the software world shortly after I left college.

Recently I resumed playing around with electronics as a hobbyist and have been working on STM23F4 boards to get my hands dirty so to speak, and although I can get stuff done in C, it is nagging at me, just how much better a language could be, should be, for this kind of work.

So I could start by asking what people's views are here, what do you - as an MCU developer - find good or bad about the prominent languages, what was a language you used in the past that had features you'd like to see today when working with MCUs?

This is informal, although I do have ideas of my own and have a draft list of goals for such a thing, I am interested in what others have to say about this subject, I'm also drawn to leveraging LLVM but know very little about it, it's generic intermediate IR is clearly helpful here as that reduces the back end effort substantially.





Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 22, 2022, 06:12:20 pm
C is fine for embedded. So far all attempts to make something better ended up in bloat and feature creep.

A slightly cleaned up version of C would be nice, but language inventors never stop at that because it is "boring".

If you absolutely can't make C work for you, it may better to figure out what you think is wrong and how others are dealing with that.
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 22, 2022, 06:16:25 pm
C is, and has always been, a hardware level oriented language. That's why it has always excelled in applications that need to be near the metal, like MCUs and operating systems. What would your new language bring to the table.
Title: Re: A new, hardware "oriented" programming language
Post by: mcovington on November 22, 2022, 06:23:30 pm
I agree - C is hardware-oriented already, which is why it lends itself to efficient use of CPUs.

What was PL/M like, and should it come back?  I've never used it.

Arduino language is actually rather nice, I think; it's C with minor extensions.  Maybe that's the direction to go in.  Full C++ definitely isn't.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 06:33:06 pm
C is, and has always been, a hardware level oriented language. That's why it has always excelled in applications that need to be near the metal, like MCUs and operating systems. What would your new language bring to the table.

There are several things that a new language would bring, here's a summary of the more salient:


These are the kinds of things that I've seen other people raise or complain about sometimes, things that a freshly designed language could readily accomodate.
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 22, 2022, 06:35:57 pm
I agree - C is hardware-oriented already, which is why it lends itself to efficient use of CPUs.

What was PL/M like, and should it come back?  I've never used it.

Arduino language is actually rather nice, I think; it's C with minor extensions.  Maybe that's the direction to go in.  Full C++ definitely isn't.
PL/M was a stripped down PL/1 Intel produced in the late 70s to program their small processors. PL/1 was a monster language, designed to do everything IBM could think of. PL/M extracted out a subset that was functionally comparable C, before C had become popular. Motorola made something similar called MPL. Other people went down a similar road. Its not entirely clear why everyone in the late 70s chose PL/1 as their starting point, but C killed that path in the early 80s.
Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 22, 2022, 06:38:42 pm
Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.
This is plain stupid on modern MCUs and if you think that NOPs are a valid timing technique, then I'm not sure I want to use that language.

Everything else is a nice incremental improvement. If it was there, it could be useful, but I would not say C suffers because of that.

Support for an async/await model.
There is a lot of this in the recent versions of C++. I'm not sure how useful it actually is as a language feature.

Making a language is a lot of effort, and I don't think stuff above justifies it. You can do it for fun, but in that case don't make polls, start doing it and if you have something useful implemented, we can discuss.

Right now this has a vibe of "making your own high performance oscilloscope" threads from 10 years ago. Those went for pages and pages with people daydreaming about it and not actually doing anything.
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 22, 2022, 06:40:17 pm
Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.
That one is certainly not going to happen. Very few machines now permit cycle exact execution of anything. Even really small devices have at least some elementary kind of cache, that messes with the instruction timing.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 06:45:19 pm
I agree - C is hardware-oriented already, which is why it lends itself to efficient use of CPUs.

What was PL/M like, and should it come back?  I've never used it.

Arduino language is actually rather nice, I think; it's C with minor extensions.  Maybe that's the direction to go in.  Full C++ definitely isn't.
PL/M was a stripped down PL/1 Intel produced in the late 70s to program their small processors. PL/1 was a monster language, designed to do everything IBM could think of. PL/M extracted out a subset that was functionally comparable C, before C had become popular. Motorola made something similar called MPL. Other people went down a similar road. Its not entirely clear why everyone in the late 70s chose PL/1 as their starting point, but C killed that path in the early 80s.

Well PL/I was influential but was heavily sidelined by the growth in popularity of C in universities. Full PL/I is indeed a large language, but it was revised into a Subset G standard (an ANSI standard) that's a smaller language, smaller than many modern languages like ADA (recently adopted by Nvidia for their firmware language of choice).

PL/I was also one of the first languages used to write an operating system, some years before C arose. Contrary to popular belief too, C was created primarily for ease of implementing a compiler, programming hardware was never a goal of the designers any more than several other languages at the time.
Title: Re: A new, hardware "oriented" programming language
Post by: jpanhalt on November 22, 2022, 06:49:57 pm
I can only write Microchip Assembly and thought that was pretty hardware oriented.

In accord with ataradov's comment, using NOP's to pad a routine to an exact time may be OK, so long as only a few are used.  For longer delays there are other ways, not the least of which is to use a timer, e.g., TMR2, to set a beat.  With such a beat, one can do other things during any appreciable delay, yet keep that time-critical routine exact -- or almost so. 
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 06:50:02 pm
Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.
That one is certainly not going to happen. Very few machines now permit cycle exact execution of anything. Even really small devices have at least some elementary kind of cache, that messes with the instruction timing.

That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C. There are assembler MCU developers out there that struggle to use C because of this kind of thing. They have carefully crafted code where the want some execution path to take exactly the same number of clock cycles as some other, so they embed multiple NOPs sometimes, their designs require that.

These are legitimate needs arising from their problem domain, enabling these to be met by supporting it in a language is surely fully appropriate for a language designed for hardware programming.


Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 06:52:44 pm
I can only write Microchip Assembly and thought that was pretty hardware oriented.

In accord with ataradov's comment, using NOP's to pad a routine to an exact time may be OK, so long as only a few are used.  For longer delays there are other ways, not the least of which is to use a timer, e.g., TMR2, to set a beat.  With such a beat, one can do other things during any appreciable delay, yet keep that time-critical routine exact -- or almost so.

Yes, I was chatting to a very experienced 8 bit PIC developer recently about this, he tries to use C for some parts of the code only to find he has no control over these hardware characteristics any more. The same thing is true of computed goto, C has no support for this yet it is trivial in assembly programming and much faster than using arrays of function pointers.
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 22, 2022, 06:53:56 pm
PL/I was also one of the first languages used to write an operating system, some years before C arose. Contrary to popular belief too, C was created primarily for ease of implementing a compiler, programming hardware was never a goal of the designers any more than several other languages at the time.
PL/1 was quite late to the "writing operating systems in a high level language" game. Which is why so few were written in it. Several languages, often described as real time oriented rather than operating system oriented, were developed for building operating systems before PL/1 existed.

C was intended to be close to the metal. That's really all a systems oriented language is.
Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 22, 2022, 06:54:05 pm
That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C.
You are very wrong here. And I think this alone is enough to stop discussion of the language design.

You understand that even on simplest MCUs NOP timing is affected by flash wait states?  This is not taking into account micro architectural optimizations, out of order execution, multiple issue pipelines, etc
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 07:02:51 pm
That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C.
You are very wrong here. And I think this alone is enough to stop discussion of the language design.

You understand that even on simplest MCUs NOP timing is affected by flash wait states?  This is not taking into account micro architectural optimizations, out of order execution, multiple issue pipelines, etc

That's rather odd, since you wrote this in January:

Here is my standard way of generating small blocking delays:

Code: [Select]
__attribute__((noinline, section(".ramfunc")))
void delay_cycles(uint32_t cycles)
{
  cycles /= 4;

  asm volatile (
    "1: sub %[cycles], %[cycles], #1 \n"
    "   nop \n"
    "   bne 1b \n"
    : [cycles] "+l"(cycles)
  );
}

Code is located in SRAM so that execution time is not subject to flash wait states.

This is better than a plain C loop, since it does not depend on the compiler and optimization settings.

Anyway, if you have nothing to contribute then of course, you can stop discussing this subject.
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 22, 2022, 07:05:22 pm
Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.
That one is certainly not going to happen. Very few machines now permit cycle exact execution of anything. Even really small devices have at least some elementary kind of cache, that messes with the instruction timing.

That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C. There are assembler MCU developers out there that struggle to use C because of this kind of thing. They have carefully crafted code where the want some execution path to take exactly the same number of clock cycles as some other, so they embed multiple NOPs sometimes, their designs require that.

These are legitimate needs arising from their problem domain, enabling these to be met by supporting it in a language is surely fully appropriate for a language designed for hardware programming.
It is very much true. Something like a PIC or an 8051 can run cycle exact code, and developers have gotten very used to that. Look at more modern small machines and you'll see them gradually drifting away from cycle exact operation. Even chips without an explicit cache to play with the timing often now have longish memory lines, which are demuxed in a non-deterministic way, to mitigate the slow operation of flash. Often people used to cycle exact operation struggle to adapt to this, and produce some very quirky results.

One area where there is some merit in cycle exact code is security. If you implement something like an encryption algorithm, its much less easy to attack it if all the paths through it are of equal length. You can't make them run in equal time every time, but if you can make their averages comparable you have made the code a lot more robust against attacks.

Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 22, 2022, 07:08:19 pm
Code is located in SRAM so that execution time is not subject to flash wait states.
Yes, and I had to account for the flash execution by placing the code in SRAM. And I knew exactly the platform this code would be running on, so I knew what optimizations would be performed and I knew that they would be predictable.

This code would not work at all on Cortex-A5 core, for example.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 07:22:15 pm
Code is located in SRAM so that execution time is not subject to flash wait states.
Yes, and I had to account for the flash execution by placing the code in SRAM. And I knew exactly the platform this code would be running on, so I knew what optimizations would be performed and I knew that they would be predictable.

This code would not work at all on Cortex-A5 core, for example.

Of course, one should always account for all factors like this in any design, I never suggested otherwise nor did I suggest it would be appropriate for all target platforms. The facts are the facts, today one must do as you did, embed assembler, all I'm suggesting here is that you'd have another option, one that C itself doesn't afford you today, the list I gave is just a starting point for discussion.
Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 22, 2022, 07:28:50 pm
What is the point of the language feature that won't be even theoretically possible to implement on a lot of modern hardware?

You don't need to embed the assembler, it is just a quick hack. The correct way of making delays is to use the timer. Again, not a compiler feature.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 07:33:08 pm
Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.
That one is certainly not going to happen. Very few machines now permit cycle exact execution of anything. Even really small devices have at least some elementary kind of cache, that messes with the instruction timing.

That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C. There are assembler MCU developers out there that struggle to use C because of this kind of thing. They have carefully crafted code where the want some execution path to take exactly the same number of clock cycles as some other, so they embed multiple NOPs sometimes, their designs require that.

These are legitimate needs arising from their problem domain, enabling these to be met by supporting it in a language is surely fully appropriate for a language designed for hardware programming.
It is very much true. Something like a PIC or an 8051 can run cycle exact code, and developers have gotten very used to that. Look at more modern small machines and you'll see them gradually drifting away from cycle exact operation. Even chips without an explicit cache to play with the timing often now have longish memory lines, which are demuxed in a non-deterministic way, to mitigate the slow operation of flash. Often people used to cycle exact operation struggle to adapt to this, and produce some very quirky results.

One area where there is some merit in cycle exact code is security. If you implement something like an encryption algorithm, its much less easy to attack it if all the paths through it are of equal length. You can't make them run in equal time every time, but if you can make their averages comparable you have made the code a lot more robust against attacks.

Yes, this subject of equal length paths came up. There was an example:

Code: [Select]

if (A | ^B & C | ^D)
{
    // do stuff
}
else
{
   // do other stuff
}


Something like that anyway. The problem is that the C compiler will use short circuit evaluation when evaluating the boolean expression, but his requirement was that the method (which contained more less just this code) must take exactly the same number of clock cycles for all possible values of A, B, C and D - that is simply impossible in C, the language itself does not offer such a feature.

I speculate that it might be feasible to somehow achieve this by either providing a way to stop short circuit evaluations or guide the compiler in some way, perhaps

Code: [Select]
exact void some_function()
{

if (A | ^B & C | ^D)
{
    // do stuff
}
else
{
   // do other stuff
}

}

This is rather contrived, but you get the idea, now the effort involved might be high, but I've not really analyzed it, it might well be possible to deliver something like that - bearing in mind the realities that ataradov pointed out.


Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 07:44:17 pm
What is the point of the language feature that won't be even theoretically possible to implement on a lot of modern hardware?

You don't need to embed the assembler, it is just a quick hack. The correct way of making delays is to use the timer. Again, not a compiler feature.

The point would be that there are use cases, situations, problems and platforms where it is an extremely helpful solution for the developer. C offers flexible memory allocation and freeing as you know, yet that, right there, is an example of a language feature that is used with great caution, if at all, on many MCU designs.

The correct way to do something is also rather subjective ataradov, as an engineer yourself, you'll appreciate that all designs are compromises.

Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 22, 2022, 07:57:01 pm
The beauty of C is that I can switch platforms with relative ease. If your language would only be helpful on some limited (and ever shrinking) subset of platforms, then it would be too dangerous to rely on it.

I just don't feel like this solves a big enough problem to justify a whole new language.

Just to be clear, I'm not stopping you, go ahead. Making compilers is fun, I've made a few myself. Just don't expect to come up with something really useful that would displace existing tools. This is very hard to do. And in this respect, dreaming of features is a waste of time. Start making a C-like compiler first and see how fast you lose interest. If you don't lose interest, then you will have a base where you can add features and then there will be something to discuss.

You are not the first one to want to do something like this, but it always ends up in just discussions. Don't ask us what you want to do, if you have a vision, implement it and then we can say if it makes sense or not.
Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 22, 2022, 08:10:20 pm
Also, have a look at existing languages before designing your own. Literally hundreds out there already. You're not the only one having tried.

For something close to C, you could have a look at Zig. See what it brings.

Before writing a compiler, try writing "real code" in your new language, just to see if it makes sense. Not just trivial code chunks.

Also, if you really want to try, I'd suggest "transpiling" to C first. Don't bother with the low-level code generation. At least not until it makes sense to do so. Additional benefit, apart from the saved time and effort, is that it will make interfacing with actual C code trivial.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 08:24:11 pm
Also, have a look at existing languages before designing your own. Literally hundreds out there already. You're not the only one having tried.

For something close to C, you could have a look at Zig. See what it brings.

Before writing a compiler, try writing "real code" in your new language, just to see if it makes sense. Not just trivial code chunks.

Also, if you really want to try, I'd suggest "transpiling" to C first. Don't bother with the low-level code generation. At least not until it makes sense to do so. Additional benefit, apart from the saved time and effort, is that it will make interfacing with actual C code trivial.

I agree with you. I've not looked at Zig actually, so thanks, I will indeed take a look. I did look at Hare (https://harelang.org/) recently another new language. I'm certainly not starting on a compiler, not yet anyway, that's a big undertaking even if using LLVM to manage the back end.

Transpiling is already on my list of things to think about, being able to generate standard looking C that performed the same operation as the new language is a great way to evaluate things.

The grammar I have started to outline resembles C only slightly, there are grammatical traits in C that are problematic not least of which is its odd way mixing procedures and functions into some hybrid concept with the useless "void" keyword and the ability to invoke a function yet overlook its return value and so on.

The use of things like ++X and J++ and so on, effectively expressions and assignments combined, reduces one's ability to reason about the code sometimes.

Then we have "forward declarations" and reliance on reserved words and so on, so there are a few things in C that are useful like braces for block delimiters and simplified structure definitions, but not a lot else.

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 08:29:11 pm
This is the kind of thing I like:

https://ziglang.org/documentation/master/#Vectors

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 08:48:36 pm
Hmm, a very thought provoking real-world comparison of Rust and Zig (https://kevinlynagh.com/rust-zig/).

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 09:33:27 pm
PL/I was also one of the first languages used to write an operating system, some years before C arose. Contrary to popular belief too, C was created primarily for ease of implementing a compiler, programming hardware was never a goal of the designers any more than several other languages at the time.
PL/1 was quite late to the "writing operating systems in a high level language" game. Which is why so few were written in it. Several languages, often described as real time oriented rather than operating system oriented, were developed for building operating systems before PL/1 existed.

C was intended to be close to the metal. That's really all a systems oriented language is.

The guys that eventually developed C worked on the Multics project. That was written in PL/I (a simplified version crafted by the late Bob Freiburghouse (https://archive.computerhistory.org/resources/access/text/2013/03/102743157-05-01-acc.pdf)).

The PL/I language brought together a number of emerging language features for the first time, some of which were founded on systems programming on large mainframes. That meant the language designers were familiar with interrupts, efficient layout of memory for complex, fine grained data structures, list processing and other things that were barely considered before.

The language was a major milestone in computing, the designers took what they regarded as the best features of Cobol, Fortran and Algol, at the time the leading languages in business, science and universities.

It was the first language to incorporate concurrency features, it had a bit data type (for the very reason that bit fields are prevalent in operating system data structures).

It was perhaps the first programming language to be specified in a formal language, the Vienna Definition Language, this allowed both the grammar and the semantics to be specified jointly.

It was certainly very influential but not so much in the world of PCs, of course CP/M an early OS for the 8080 family by Bill Gate's nemesis Gary Kildall the founder of Digital Research. CP/M was written (this is an 8 bit OS in the 1970s, talk about resource constrained hardware) in a variant of PL/I named PL/M, by Kildall.

Consider:

Quote
Unlike other contemporary languages such as Pascal, C or BASIC, PL/M had no standard input or output routines. It included features targeted at the low-level hardware specific to the target microprocessors, and as such, it could support direct access to any location in memory, I/O ports and the processor interrupt flags in a very efficient manner. PL/M was the first higher level programming language for microprocessor-based computers and was the original implementation language for those parts of the CP/M operating system which were not written in assembler. Many Intel and Zilog Z80-based embedded systems were programmed in PL/M during the 1970s and 1980s. For instance, the firmware of the Service Processor component of CISC IBM AS/400 was written in PL/M.

The C language became popular not because of any inherent language prowess, not because of some special affinity for "hardware" level coding, but because there was no commercially available PL/I implementation available for early PCs and Microsoft had never created a compiler for it (though the did sell COBOL and FORTRAN compilers back then). When MS DOS started to dominate rather than CP/M there was no huge motive to invest in PL/I and so C started to grow because (and this is the chief reason) it is very easy to write a compiler for C. So it is an accident of history and market forces that elevated C to prominence and then of course Borland competed with Microsoft with C and then C++ emerged, leading to more attention for C.

Too few developers stop to learn the history of such things, PL/I was not the perfect language, none are, but it did bring together some (at the time) revolutionary ideas and some of those are still unknown in modern languages, technology is as influenced by fashion as many other things in life!



Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 22, 2022, 09:41:31 pm
The C language became popular not because of any inherent language prowess, (...)

That can be said for almost all languages that became popular.
They all became popular because they filled some void.

C has passed the test of time though, and I'm not sure all of those new languages will.

Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 22, 2022, 10:12:46 pm
The C language became popular not because of any inherent language prowess, not because of some special affinity for "hardware" level coding, but because there was no commercially available PL/I implementation available for early PCs and Microsoft had never created a compiler for it (though the did sell COBOL and FORTRAN compilers back then). When MS DOS started to dominate rather than CP/M there was no huge motive to invest in PL/I and so C started to grow because (and this is the chief reason) it is very easy to write a compiler for C. So it is an accident of history and market forces that elevated C to prominence and then of course Borland competed with Microsoft with C and then C++ emerged, leading to more attention for C.

Too few developers stop to learn the history of such things, PL/I was not the perfect language, none are, but it did bring together some (at the time) revolutionary ideas and some of those are still unknown in modern languages, technology is as influenced by fashion as many other things in life!
PL/M was released really early in the life of microprocessors - around 1974.  C is listed as being earlier than that, but that was just internal to Bell Labs. It didn't get a real public viewing for a while. When I learned C at the end of the 70s it was still niche, while huge amounts of code for microprocessors existed in PL/M. This included CP/M itself and a lot of early CP/M applications. I was using a mature implementation of PL/M86 in 1979, when the first 8086s were available, and before MSDOS was. It had been developed and tuned with an 8086 simulator, and we did some of our development using that simulator, because the 8086 chips were not yet widely available. For quirky historical reasons the developer of MSDOS, before MS bought it, developed it in assembly language.  Since PL/M86 was already available, and MSDOS was developed roughly as a CP/M clone, a lot of existing code could easily have been ported to MSDOS, but I'm not clear how much was. Microsoft misjudged the market, and put effort into Fortran and Cobol. They had to buy in a decent C compiler from Lattice when C suddenly took off. I don't think its clear why PL/M suddenly died, but it could have been Intel's licencing fees. The PC suddenly made cheap king in the software market. For people like me, who had used Algol, Coral, PL/1, PL/M and other languages for several years, C was just what we were looking for.

Fortran on microprocessors wasn't that commercially valuable in the early days of MSDOS. Cobol was, but MS-COBOL was a dog. Things like MicroFocus's MicroCOBOL dominated the "lets move this app to a small machine" market.

You didn't mention the biggest influence on C, and the language from which its name comes - BCPL. BCPL and C are about as unlike PL/1 in their thinking as could be.

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 22, 2022, 11:03:03 pm
The C language became popular not because of any inherent language prowess, not because of some special affinity for "hardware" level coding, but because there was no commercially available PL/I implementation available for early PCs and Microsoft had never created a compiler for it (though the did sell COBOL and FORTRAN compilers back then). When MS DOS started to dominate rather than CP/M there was no huge motive to invest in PL/I and so C started to grow because (and this is the chief reason) it is very easy to write a compiler for C. So it is an accident of history and market forces that elevated C to prominence and then of course Borland competed with Microsoft with C and then C++ emerged, leading to more attention for C.

Too few developers stop to learn the history of such things, PL/I was not the perfect language, none are, but it did bring together some (at the time) revolutionary ideas and some of those are still unknown in modern languages, technology is as influenced by fashion as many other things in life!
PL/M was released really early in the life of microprocessors - around 1974.  C is listed as being earlier than that, but that was just internal to Bell Labs. It didn't get a real public viewing for a while. When I learned C at the end of the 70s it was still niche, while huge amounts of code for microprocessors existed in PL/M. This included CP/M itself and a lot of early CP/M applications. I was using a mature implementation of PL/M86 in 1979, when the first 8086s were available, and before MSDOS was. It had been developed and tuned with an 8086 simulator, and we did some of our development using that simulator, because the 8086 chips were not yet widely available. For quirky historical reasons the developer of MSDOS, before MS bought it, developed it in assembly language.  Since PL/M86 was already available, and MSDOS was developed roughly as a CP/M clone, a lot of existing code could easily have been ported to MSDOS, but I'm not clear how much was. Microsoft misjudged the market, and put effort into Fortran and Cobol. They had to buy in a decent C compiler from Lattice when C suddenly took off. I don't think its clear why PL/M suddenly died, but it could have been Intel's licencing fees. The PC suddenly made cheap king in the software market. For people like me, who had used Algol, Coral, PL/1, PL/M and other languages for several years, C was just what we were looking for.

Fortran on microprocessors wasn't that commercially valuable in the early days of MSDOS. Cobol was, but MS-COBOL was a dog. Things like MicroFocus's MicroCOBOL dominated the "lets move this app to a small machine" market.

You didn't mention the biggest influence on C, and the language from which its name comes - BCPL. BCPL and C are about as unlike PL/1 in their thinking as could be.

Yes, they are indeed extremely different. Bob Freiburghouse was an interesting guy, he accidentally got into computing after graduating in EE (as his bio explains) and somehow got asked to write a Fortran compiler for some machine or other, perhaps while still in the navy. Anyway he went to to setup a language business in the 60s, he did well, he supplied several language compilers to DEC, CDC, Burroughs and several others for some years, this included variants of PL/I.

He describes how he developed a generic table driven code generator that allowed him to take any of his front end and generate the code for any back end, this was highly innovative at the time, when few companies were in the commercial language business.

He also worked on Multics, here's a fascinating paper (https://multicians.org/pl1-raf.html) of his I used as a guide when developing a PL/I compiler for Windows, I recall getting a copy of this from a library in London, this was before the internet.

This other paper (https://multicians.org/pl1.html) explains how PL/I came to be selected for Multics development.

There are a handful of things I noticed when I first began to learn C back in 1990, that are the same as in PL/I, these are 1) Comments use /* and */ and 2) pointer referencing uses -> and 3) The dot notation for structure member access, these all originated (I believe) in PL/I, which Kernigan and Ritchie must have used when working on Multics.









Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 22, 2022, 11:34:55 pm
posts like this are one of the reasons why i don't want to publish or release my-c.
Yet you can't stop mentioning it in every topic.
Title: Re: A new, hardware "oriented" programming language
Post by: DiTBho on November 22, 2022, 11:45:24 pm
There are a handful of things I noticed when I first began to learn C back in 1990

yup, indeed the first release Dragon Book offers PL/I examples mixed with C-examples.
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 23, 2022, 01:38:57 am
C is, and has always been, a hardware level oriented language. That's why it has always excelled in applications that need to be near the metal, like MCUs and operating systems. What would your new language bring to the table.

There are several things that a new language would bring, here's a summary of the more salient:

  • No reserved words, thus enabling new keywords to be added over time.

A recipe for programmer confusion, and nothing whatsoever to do with MCUs

Quote
  • Support 'bit' as a native data type.
  • Support 'strings' as a native type, BCD/decimal as well.

Not native types on many or most MCUs, so not sure what you're trying to achieve there. What kind of strings? That sounds like you need a heap.

Quote
  • Support for namespaces.

Nothing whatsoever to do with MCUs. C++ already exists.

Quote
  • Computed gotos.

You want to calculate the actual machine address to jump to? In a portable high level language? Presumably with equal spacing between the different jump targets? So the programmer is going to have to know the exact instructions on the target that will be generated from each bit of code being jumped to, and their size?

Insanity.

Quote
  • Flexible alignment, packing and padding directives.
  • Nested functions.

C already has this, at least as implemented in gcc and clang. Most MCUs don't have any better support for inserting and extracting arbitrary bit fields than C has, so even if you can write it a bit more simply you are fooling yourself about the efficiency. And you can already use macros or inline functions in C.

Quote
  • Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.

Most modern CPUs, including microcontrollers, can't guarantee that without taking significant other measures that the programmer really has to arrange by themselves, such as loading critical code into TIM instead of flash, turning off caches and branch prediction, and so forth.

This is not an appropriate feature for a supposedly high level language.

Quote
  • Support for an async/await model.

This requires heap-allocated closures, and I think probably garbage collection.

On a PIC? Just no.
Title: Re: A new, hardware "oriented" programming language
Post by: mcovington on November 23, 2022, 02:45:50 am
I never used PL/M but remember PL/1 very fondly, having used it a lot in the mid to late 1970s.  It was the first programming language we encountered that did not actively try to restrict what we could do, and it suited IBM mainframes very well (that's what a lot of us were using).  Most of us didn't see C until the 1980s.
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 23, 2022, 04:38:16 am
I never used PL/M but remember PL/1 very fondly, having used it a lot in the mid to late 1970s.  It was the first programming language we encountered that did not actively try to restrict what we could do, and it suited IBM mainframes very well (that's what a lot of us were using).  Most of us didn't see C until the 1980s.

I used PL/I in my first job, on a DG minicomputer. Some subset I guess. It was like Pascal, FORTRAN, and COBOL thrown into a big heap and thoroughly stirred.
Title: Re: A new, hardware "oriented" programming language
Post by: Siwastaja on November 23, 2022, 07:52:58 am
A slightly cleaned up version of C would be nice, but language inventors never stop at that because it is "boring".

+1 for this, best chance for success is to take C and make small improvements to it, making it truly hardware-oriented. For example, you could add keywords like big_endian or little_endian so that compiler would handle the conversions and interfaces would look clean with no manual shifting, make bitfield ordering well defined, maybe add ranges to variables ADA/VHDL-style which would enable the compiler to do compile-time checking and optimize better.

And of course, arbitrary width types. Maybe I want uint5_t and int129_t. As you are building a new language, they should not be defined type names in some header file like in C, but a type system, syntax could look like uint(8) or something.

Title: Re: A new, hardware "oriented" programming language
Post by: IDEngineer on November 23, 2022, 08:23:32 am
The idea of the compiler calculating NOP's to yield equal execution time through multiple code paths sounds great until you remember interrupts, which are almost always a part of embedded systems.

What are you going to do, disable interrupts for large portions of code to prevent asynchronicity? That defeats the very purpose of hardware interrupts and essentially forces a polling based architecture that won't be compatible with a huge number of embedded environments.

I suppose you could commandeer some of the target's hardware timer resources and set up a sort of timekeeping scheme, but that smacks of certain dynamic "features" which are the worst parts of C++ and many other OO-like languages.

I'm not discouraging you from the exercise, but have realistic expectations. The chance for something like this to go mainstream is very close to zero.
Title: Re: A new, hardware "oriented" programming language
Post by: JPortici on November 23, 2022, 09:20:05 am
...

wouldn't have said it better.
Mentioning PICs, some of the proposed improvements are already part of the XC compilers, others are requests that clearly come from assembly programmers who can't stop thinking in assembly and ostentably write in what i call "Clever C" (forgetting that you CAN, and some time must, use separate assembly modules. Just conform to the ABI and call them as C functions so the compiler is happy, other programmers will be as well)
Dedicated strings requrie heap and personally speaking in a C project i prefer to avoid heap. In languages in which is a given i simply apply another mindset (using it responsibly of course)
Title: Re: A new, hardware "oriented" programming language
Post by: Siwastaja on November 23, 2022, 10:54:34 am
That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C. There are assembler MCU developers out there that struggle to use C because of this kind of thing. They have carefully crafted code where the want some execution path to take exactly the same number of clock cycles as some other, so they embed multiple NOPs sometimes, their designs require that.

Simple enough microcontrollers that can produce cycle-accurate predictable timing by simple instruction counting are becoming exceedingly rare. These 8-bitters still exist, but such cycle-accuracy combined to generally very crappy performance is something people do not actually want. Instead, people buy modern high-performance microcontrollers which can still produce very accurate timing by just scaling the absolute time used per cycle down, by utilizing higher clock speed. For example, a 12-cycle ISR latency on a 400MHz MCU looks like HALF a cycle on a 16MHz AVR/PIC; at that point you simply do no care if it sometimes takes 14 cycles due to some pipeline or branch mispredict or something. I have done pretty timing-sensitive things simply on interrupt handlers, and the advantage is ease of writing, reading, and maintaining that code. Manual cycle counting, or automated version thereof, is simply not needed anymore, except in very rare cases, which require careful understanding anyway.

The problem with this "predictive timing language" is, it becomes tied to certain type of hardware, and then you have replicated what the XCORE folks have done (apparently pretty well).
Title: Re: A new, hardware "oriented" programming language
Post by: Bicurico on November 23, 2022, 11:52:56 am
I am not an MCU programmer.

Why not? And why am I posting here?

Well, I am not an MCU programmer, because for my needs I don't need it and for hobby purposes it is simply said too complex for the occasional project.

I am posting here, to add a different perspective:

In my opinion, the greatest revolution in MCU programming was due to Arduino and the extremely easy and well documented SDK. This opened doors to many hobby applications, otherwise inacessible to a wide percentage of hobbyists.

So I wonder: why try to program "Yet Another C Compiler" for MCU, if the existing programmers have no problem with the existing SDK's (as the discussion so far indicates)? Why not, instead, develop an SDK/language for the masses that allow to use more generic (cheaper) MCU and program them as easily (or even more easily) as what Arduino offers?

Also, another suggestion: current MCU SDK's are bloated in size! If you want to program PIC, AVR and something else, you easily fill your harddisk with three huge installations. If you want to do something nice, make it small! Preferrably something that just deflates into a single folder.

Sorry if this is beyond your purpose or if my line of thought doesn't make sense (it does to me, though).

Regards,
Vitor
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 23, 2022, 12:28:19 pm

There are several things that a new language would bring, here's a summary of the more salient:

  • No reserved words, thus enabling new keywords to be added over time.
  • Support 'bit' as a native data type.
  • Support 'strings' as a native type, BCD/decimal as well.
  • Support for namespaces.
  • Computed gotos.
  • Flexible alignment, packing and padding directives.
  • Nested functions.
  • Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.
  • Support for an async/await model.

These are the kinds of things that I've seen other people raise or complain about sometimes, things that a freshly designed language could readily accomodate.

Items to consider...

Cache coherency, between different cores and with interrupts. That implies a memory model, as C has belatedly realised.

Non uniform memory architecture. Some bits of memory are much further away from the cores than others. "Caches are the new RAM, RAM is the new disc, disc is the new mag tape".

If you want guaranteed timing, then consider the effects of all the caches (inc TLB and others). In addition, you must understand what was necessary to achieve guaranteed timing. See the XMOS xCORE plus xC system (Buy them at digikey ;) ) That also has the equivalent of your async/await sorted in a sound theoretical and practical way, which has since been copied in other languages.the

Rather than have a computed goto, have a computed come from instruction. Yes, that has existed!
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 23, 2022, 01:11:55 pm
In my opinion, the greatest revolution in MCU programming was due to Arduino and the extremely easy and well documented SDK. This opened doors to many hobby applications, otherwise inacessible to a wide percentage of hobbyists.

This I entirely agree with.

For many people and many purposes, it is completely appropriate to use a more powerful MCU than you strictly need, and use some of that extra speed and memory capacity to run easier to use programming languages and libraries.

The Arduino library, implemented using C++ for a 16 MHz AVR, is a good step in that direction.

Some people are aghast that digitalWrite() takes 2 µs when you can do the "same thing" (not quite, actually) using a single AVR machine code instruction taking 1/16th of a µs.

Which matters not at all if you're using it to switch on your garage light, or even control a motor or R/C aircraft servo where a control loop of 1 ms or even 10 ms is absolutely fine.

And if you run the Arduino library on a 320 MHz HiFive1 or 600 MHz dual-issue Teensy 4.0 or 240 MHz ESP32 then your inefficient code runs just as well as the finest hand-crafted AVR or PIC code.

The problem then becomes that C++ and the Arduino library are too low level and inconvenient for many things. Python or JavaScript or Scheme becomes a much better idea.

And then you can have convenient strings and dictionaries and async/await programming model.

I'm not against any of that.

I'm against thinking you can or should try to mix that with using NOPs for timing, or trying to force branches of a switch to have equal execution times. JUST NO. Use a faster CPU and proper event-driven programming.

We now have an amazing situation where a 20 MHz 8 bit 2 KB RAM ATMega328 costs $2.50 in quantity, but you can get a 200 MHz 32 bit RISC-V MCU with WIFI and BlueTooth and several hundred KB of RAM for about the same price. Or, you can get a 48 MHz 32 bit RISC-V with 4 KB RAM for $0.12.

Quote
Also, another suggestion: current MCU SDK's are bloated in size! If you want to program PIC, AVR and something else, you easily fill your harddisk with three huge installations.

I don't know why people want customised Eclipse-based IDEs, or why manufacturers think they do. Command line toolchains are not very big.
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 23, 2022, 01:14:50 pm
Cache coherency, between different cores and with interrupts. That implies a memory model, as C has belatedly realised.

Basically as soon as dual-core PCs started to be common.
Title: Re: A new, hardware "oriented" programming language
Post by: mcovington on November 23, 2022, 01:39:42 pm
For many people and many purposes, it is completely appropriate to use a more powerful MCU than you strictly need, and use some of that extra speed and memory capacity to run easier to use programming languages and libraries.

HEAR, HEAR!  It is entirely appropriate to use a $20 microcontroller instead of a 50-cent microcontroller in order to greatly facilitate the development of the firmware.  In a mass-produced product with simple firmware, use the 50-cent micro, and spend a month writing the program.  But for a one-off prototype (whether in industry, science, or hobby), use the Arduino or RPi and program it in a language that is easy to use and test.

If it were $20 versus $500, it would be a somewhat different trade-off.

I think there was such an issue with mainframes 50 years ago, but not so well understood.  The IBM 370 and OS were designed to run the apps, with a lot of fine tuning of disk performance, etc., and were harder to develop software on.   (IEFBR14, for those who remember it, was the reductio ad absurdum of that.)   VAX/VMS and UNIX seemed to be slanted much more towards making life easier for the programmers, on the ground that if you do that, they will make the apps run better.  And look at what almost all subsequent programming is descended from!
Title: Re: A new, hardware "oriented" programming language
Post by: newbrain on November 23, 2022, 01:42:23 pm
And of course, arbitrary width types. Maybe I want uint5_t and int129_t. As you are building a new language, they should not be defined type names in some header file like in C, but a type system, syntax could look like uint(8 ) or something.
Your wish has been graciously granted by the standard fairy, including similar syntax. (https://open-std.org/JTC1/SC22/WG14/www/docs/n3054.pdf#subsection.6.2.5)
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 23, 2022, 02:14:30 pm
I'm against thinking you can or should try to mix that with using NOPs for timing, or trying to force branches of a switch to have equal execution times. JUST NO. Use a faster CPU and proper event-driven programming.

Seconded.

However the issue of predictable and predicted timing is important when considering whether deadlines will be met.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 23, 2022, 02:17:34 pm
Cache coherency, between different cores and with interrupts. That implies a memory model, as C has belatedly realised.

Basically as soon as dual-core PCs started to be common.

It was a practical problem decades earlier. As usual, the HPC mob ran into the problem earlier than most, but commercial computing wasn't far behind.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 03:47:34 pm
...

wouldn't have said it better.
Mentioning PICs, some of the proposed improvements are already part of the XC compilers, others are requests that clearly come from assembly programmers who can't stop thinking in assembly and ostentably write in what i call "Clever C" (forgetting that you CAN, and some time must, use separate assembly modules. Just conform to the ABI and call them as C functions so the compiler is happy, other programmers will be as well)
Dedicated strings requrie heap and personally speaking in a C project i prefer to avoid heap. In languages in which is a given i simply apply another mindset (using it responsibly of course)

I've only mentioned the kinds of things raised by very experienced hardware/software engineers that rely heavily on PIC, their needs, concerns and frustrations are real, not imagined, I am simply collating broad issues and concerns raised by people who find C restrictive or inflexible.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 04:29:45 pm
A slightly cleaned up version of C would be nice, but language inventors never stop at that because it is "boring".

+1 for this, best chance for success is to take C and make small improvements to it, making it truly hardware-oriented. For example, you could add keywords like big_endian or little_endian so that compiler would handle the conversions and interfaces would look clean with no manual shifting, make bitfield ordering well defined, maybe add ranges to variables ADA/VHDL-style which would enable the compiler to do compile-time checking and optimize better.

And of course, arbitrary width types. Maybe I want uint5_t and int129_t. As you are building a new language, they should not be defined type names in some header file like in C, but a type system, syntax could look like uint(8) or something.

That is a possibility, improving C. However that does impose restrictions on grammar and for me - as a compiler/language developer - the grammar is perhaps the most interesting area, once decisions have been made at a grammar level there's usually no going back, one must live with the consequences for all future versions.

What you say about arbitrary width types is something I value a great deal, of course PL/I had that and the grammar fully supports it. There's a lot to be gained by leveraging a grammar like that.

I have a draft grammar that I'm outlining and the way we declare items is a key part of that, there are problems with the C practice of

Code: [Select]
<typename> <identifier> ;
as the basis for declarations, the biggest of these IMHO is that it restricts language enhancements because the <typename> could be a user defined name like say "await" or "yield" and then there, right there, we kill any prospect of ever adding a new language keyword "await" (however that might be implemented, which is an unrelated question here).

So keyword driven languages are an obvious choice for an extensible grammar, in the case of declaration we simply have a keyword as PL/I, Kotlin, Rust and some other languages do.

Being able to specify values for type related attributes is a huge help too, for extensibility, bin(3), bin(99) whatever, are parameterizing the concept of binary numbers and can be extended to many other types too.

If we declare a "string" in C we must indicate the length/size so why is that flexibility restricted? why can't it be used just as easily for numeric types or other types? Only the grammar matters here, once the grammar is defined there's no going back.
 


Title: Re: A new, hardware "oriented" programming language
Post by: Siwastaja on November 23, 2022, 04:30:24 pm
I've only mentioned the kinds of things raised by very experienced hardware/software engineers that rely heavily on PIC, their needs, concerns and frustrations are real, not imagined, I am simply collating broad issues and concerns raised by people who find C restrictive or inflexible.

To me, you just sound like a case of Dunning-Kruger. Those who "rely heavily on PIC" probably are "very experienced engineers" only by very flexible definition of "very experienced". Your posts have already demonstrated you having quite serious lack of basic knowledge*, probably by your learning getting hindered by excessive self-esteem. I would say: lurk moar. This forum is a great resource, read and participate in discussions on the MCU & programming subforums for 5 more years. There is a lot to learn, and by getting to read comments from some people MUCH MUCH more experienced than your PIC friends, your priority list of things you would like to see in a "C replacement" would significantly change.

*) for example, mixing totally up what "native" types mean, which is pretty important in context of hardware-oriented language!

But sure, there is a niche of cases where writing cycle accurate things on a 8-bit PIC is still relevant, and where a C replacement which can help do that with less work, is also relevant. But the result would not be "a new, hardware oriented programming language", but "a PIC-specific niche tool".

And sorry in advance for being blunt, don't take it too seriously, I just struggle to write in any other way, take it with a grain of salt.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 04:35:17 pm
The idea of the compiler calculating NOP's to yield equal execution time through multiple code paths sounds great until you remember interrupts, which are almost always a part of embedded systems.

What are you going to do, disable interrupts for large portions of code to prevent asynchronicity? That defeats the very purpose of hardware interrupts and essentially forces a polling based architecture that won't be compatible with a huge number of embedded environments.

I suppose you could commandeer some of the target's hardware timer resources and set up a sort of timekeeping scheme, but that smacks of certain dynamic "features" which are the worst parts of C++ and many other OO-like languages.

I'm not discouraging you from the exercise, but have realistic expectations. The chance for something like this to go mainstream is very close to zero.

These are absolutely important questions, they are pertaining though to language semantics and I've not devoted much time at all to that area yet. I'm primarily looking at the grammar and leaning toward a PL/I like grammar which is a simple grammar yet offers huge flexibility for future growth of the language.

The suitability of emitting umpteen NOP instructions is down to the developer, it is for them to use or not as they see fit and of course they could abuse it or misuse it just as we can do today with many languages, so I fully accept the concerns you raise but see them as the domain of the designer rather than the language itself.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 04:41:32 pm
That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C. There are assembler MCU developers out there that struggle to use C because of this kind of thing. They have carefully crafted code where the want some execution path to take exactly the same number of clock cycles as some other, so they embed multiple NOPs sometimes, their designs require that.

Simple enough microcontrollers that can produce cycle-accurate predictable timing by simple instruction counting are becoming exceedingly rare. These 8-bitters still exist, but such cycle-accuracy combined to generally very crappy performance is something people do not actually want. Instead, people buy modern high-performance microcontrollers which can still produce very accurate timing by just scaling the absolute time used per cycle down, by utilizing higher clock speed. For example, a 12-cycle ISR latency on a 400MHz MCU looks like HALF a cycle on a 16MHz AVR/PIC; at that point you simply do no care if it sometimes takes 14 cycles due to some pipeline or branch mispredict or something. I have done pretty timing-sensitive things simply on interrupt handlers, and the advantage is ease of writing, reading, and maintaining that code. Manual cycle counting, or automated version thereof, is simply not needed anymore, except in very rare cases, which require careful understanding anyway.

The problem with this "predictive timing language" is, it becomes tied to certain type of hardware, and then you have replicated what the XCORE folks have done (apparently pretty well).

This is simply not true, I spoke recently with very experienced engineers who routinely use assembler for 8 bit PIC processors that are a large market. Trying to recode some parts of their existing, working designs in C is proving a huge challenge with code bloat and optimizations causing serious problems. Simple optimizations too like short-circuit evaluations.

All I'm arguing here really is for a new language to have features that help avoid these kinds of problems, where some aspects of the generated code are more controllable in a fine grained way, the NOP idea is just an example of what's required rather than a formal proposed language feature.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 04:55:16 pm
I am not an MCU programmer.

Why not? And why am I posting here?

Well, I am not an MCU programmer, because for my needs I don't need it and for hobby purposes it is simply said too complex for the occasional project.

I am posting here, to add a different perspective:

In my opinion, the greatest revolution in MCU programming was due to Arduino and the extremely easy and well documented SDK. This opened doors to many hobby applications, otherwise inacessible to a wide percentage of hobbyists.

So I wonder: why try to program "Yet Another C Compiler" for MCU, if the existing programmers have no problem with the existing SDK's (as the discussion so far indicates)? Why not, instead, develop an SDK/language for the masses that allow to use more generic (cheaper) MCU and program them as easily (or even more easily) as what Arduino offers?

Also, another suggestion: current MCU SDK's are bloated in size! If you want to program PIC, AVR and something else, you easily fill your harddisk with three huge installations. If you want to do something nice, make it small! Preferrably something that just deflates into a single folder.

Sorry if this is beyond your purpose or if my line of thought doesn't make sense (it does to me, though).

Regards,
Vitor

I couldn't agree more, the basis for my interest in this is simply that C is not and never was designed for MCU programming, it was never designed at all for that problem domain. So my questions are really about how could, how should a language be designed specifically for this problem domain? That's where I'm coming from.

I am also certainly not proposing "Yet Another C Compiler" either, the language might end up with a few superficial similarities to C but will likely have more in common with say PL/I as well as other languages.

The issue of some degree of fine grained control over code generation keeps coming up too, I don't think that is ever given much consideration at all in programming language design, the goal for decades has been to totally abstract the hardware away as much as possible, Considering ideas and options for giving some kind of control over the code generation is a novel concept.

As I said earlier the "emit multiple NOPs" idea is more by way of an example, a way to convey the problem, another way to expose that kind of control at a language level might be something like:

Code: [Select]
exact void some_function()
{

if (A | ^B & C | ^D)
{
    // do stuff
}
else
{
   // do other stuff
}

}

Where an attribute "exact" or "uniform" (for example) conveys the requirement that all execution paths should be the same - so far as is reasonably possible anyway. That is a direct influence over code generation and platform specifics, so how it would work isn't the subject (yet). But there are semantic implications immediately, for example an "exact" function could never be allowed to call or invoke any other function that was not itself "exact".




Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 23, 2022, 05:00:06 pm
I'm not sure why are you trying to convince us. if you think it is a good idea - do it. Get back to us when you have it working and we'll have a look at it.

This is not the first dreamy thread about ideal languages. There is no point in dreaming up more potential ideas until there is a base compiler that at least does something.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 05:01:24 pm
I've only mentioned the kinds of things raised by very experienced hardware/software engineers that rely heavily on PIC, their needs, concerns and frustrations are real, not imagined, I am simply collating broad issues and concerns raised by people who find C restrictive or inflexible.

To me, you just sound like a case of Dunning-Kruger. Those who "rely heavily on PIC" probably are "very experienced engineers" only by very flexible definition of "very experienced". Your posts have already demonstrated you having quite serious lack of basic knowledge*, probably by your learning getting hindered by excessive self-esteem. I would say: lurk moar. This forum is a great resource, read and participate in discussions on the MCU & programming subforums for 5 more years. There is a lot to learn, and by getting to read comments from some people MUCH MUCH more experienced than your PIC friends, your priority list of things you would like to see in a "C replacement" would significantly change.

*) for example, mixing totally up what "native" types mean, which is pretty important in context of hardware-oriented language!

But sure, there is a niche of cases where writing cycle accurate things on a 8-bit PIC is still relevant, and where a C replacement which can help do that with less work, is also relevant. But the result would not be "a new, hardware oriented programming language", but "a PIC-specific niche tool".

And sorry in advance for being blunt, don't take it too seriously, I just struggle to write in any other way, take it with a grain of salt.

I don't regard your post as blunt only rude Your personal opinion of my "understanding" and "knowledge" and "excessive self esteem" have no place in intellectual discourse. They are - if I might be so blunt - little more than ad hominem arguments, so please can we not stick to the subject under discussion and not your opinions of my character or motives or abilities?

Thank you.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 05:04:33 pm
I'm not sure why are you trying to convince us. if you think it is a good idea - do it. Get back to us when you have it working and we'll have a look at it.

This is not the first dreamy thread about ideal languages. There is no point in dreaming up more potential ideas until there is a base compiler that at least does something.

With all due respect, I am not the subject sir, the subject is in the thread's title, your opinions of me, my motives and so on are irrelevant to the subject under discussion.

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 05:08:23 pm
I have a golden rule when discussing potentially emotive subjects in internet forums, I know others likely shares this but evidently there are some who do not. That rule is that I never say anything to a person that I would not be prepared to say to them in person sitting in a meeting room with several other people. If one applies that rule before posting, the world would possibly become a better place, anyway - back to the subject, I'm enjoying the discussion, some good constructive replies have come in and they do give food for thought.


Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 23, 2022, 05:10:36 pm
Ok, but what do you want from us then?

The only question in the OP is what we think about the existing languages. And majority of people that participated in the thread seems to agree that existing languages are sufficient to the point where there is no emergency. If something better comes up, we'll have a look at it.

Nobody wants to think of cool features you can implement in some imagined language because this is not the first and not the last thread. Nothing ever comes out of them.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 05:20:04 pm
Ok, but what do you want from us then?

I see the participants here as individual and independent voices, so I would not refer to the participants as "us", you speak for yourself here, there is no "us" and to imply that is to presume everybody here agrees with your views which I don't think is true.

The only question in the OP is what we think about the existing languages. And majority of people that participated in the thread seems to agree that existing languages are sufficient to the point where there is no emergency. If something better comes up, we'll have a look at it.

Even if that were true, what of it? Disagreement - polite, sound, reasoned disagreement - is a normal and healthy part of intellectual discourse, being disagreed with is often valuable in some way or other, I certainly do not expect universal agreement in any technical discussion.

Nobody wants to think of cool features you can implement in some imagined language because this is not the first and not the last thread. Nothing ever comes out of them.

So the argument you are now making is that you can prove with absolute certainty that discussions in this forum about programming language grammars and semantics always, absolutely, in every case, never ever yield anything of utility?

That sir is a philosophical opinion not an evidence based proposition, at least I'd be very surprised indeed if you could offer a proof.





Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 23, 2022, 06:03:12 pm
...

wouldn't have said it better.
Mentioning PICs, some of the proposed improvements are already part of the XC compilers, others are requests that clearly come from assembly programmers who can't stop thinking in assembly and ostentably write in what i call "Clever C" (forgetting that you CAN, and some time must, use separate assembly modules. Just conform to the ABI and call them as C functions so the compiler is happy, other programmers will be as well)
Dedicated strings requrie heap and personally speaking in a C project i prefer to avoid heap. In languages in which is a given i simply apply another mindset (using it responsibly of course)

I've only mentioned the kinds of things raised by very experienced hardware/software engineers that rely heavily on PIC, their needs, concerns and frustrations are real, not imagined, I am simply collating broad issues and concerns raised by people who find C restrictive or inflexible.

I do hope you aren't planning a language that is specific to current PIC processors and current PIC peripherals.

If that isn't the case, what will your strategy for inclusion/exclusion of a language feature? Either a small language that is all useful with all processors/peripherals. Or a large language that contains all possibilities, only some of which are exploitable with given processor/peripherals.

Without being able to articulate your strategy (to an audience including yourself), you will flail around in different directions.
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 23, 2022, 06:28:59 pm
This is simply not true, I spoke recently with very experienced engineers who routinely use assembler for 8 bit PIC processors that are a large market. Trying to recode some parts of their existing, working designs in C is proving a huge challenge with code bloat and optimizations causing serious problems. Simple optimizations too like short-circuit evaluations.
Code bloat is pretty much inevitable when using a high level language for a PIC or an 8051. There was one very focused C compiler for tiny cores. MicroChip bought it, and shut down things like the 8051 version, which had been quite popular for putting fairly complex apps on 8051 cores. That left the market with no dense code compilers for the 8051, and a reasonably dense compiler for the PIC. If you think code bloat is bad with the PIC compiler, you should try looking at some 8051 apps.
Title: Re: A new, hardware "oriented" programming language
Post by: Siwastaja on November 23, 2022, 06:52:50 pm
This is simply not true, I spoke recently with very experienced engineers who routinely use assembler for 8 bit PIC processors that are a large market. Trying to recode some parts of their existing, working designs in C is proving a huge challenge with code bloat and optimizations causing serious problems.

(emphasis mine.) Again being rude (to other people, too:) sounds like peter-h is one of your very experienced engineers.

I can relate, I have written cycle-accurate code on 8-bitters and so on, but got into more challenging projects more than 15 years ago. I still believe what you propose is more like a niche tool than actually useful as a generic paradigm, or a necessity.

Old projects, on old HW,  written using old-time methods - let them be, or if you absolutely must change them, find people who can work like "back then". If large changes are needed, redesign using today's HW and design principles.

I'm suspicious about the need of doing large, fundamental rewrites to 8-bit projects without hardware changes, and I'm also suspicious about the supposedly large size of these 8-bit projects. They are usually quite small, given the limitations in memory.

Squeezing out the very last drops of memory or CPU performance (micro-optimization) is specialized work anyway. If you are going to sell 10000 units you just use large enough MCU and pay some dozen cents more for it and don't need to optimize until cows come home. If you are going to sell 10000000 units, then it makes financial sense to squeeze hell out of it to save $0.10 * 10000000 = $1000000, but then you can afford a real professional who does the job in a few days and does not complain about needing a new language. Been there, done that, too.
Title: Re: A new, hardware "oriented" programming language
Post by: IDEngineer on November 23, 2022, 07:12:21 pm
C is not and never was designed for MCU programming
True, and yet it's just about the perfect "universal language" for embedded applications. It's close to the hardware without (necessarily) requiring the user to learn the specifics of the chip. Bloat is often related to a language's abstraction level, after all. Just as bloat is often related to the programmer's inexperience with the underlying hardware.

As for a "language that is [processor] specific", that's what libraries are for. And that's not limited to C, either. All the common, basic constructs are part of the language whether it's C, C++, Python, Basic, etc. Then, if you don't want to roll your own interface to the specific hardware, you grab a library. Sure it will be bloated since libraries generally try to be all things to all people, but that's the price you pay for the convenience of being abstracted (there's that word again!) from the hardware details.

I suspect all you're going to get with an "embedded language" is something that looks like C with a bunch of libraries - and that exists today. And it will be specific to that one family... so you'll have different "language versions" for various processors. Even the API won't be the same because the on-chip peripherals differ from one family to another, so code won't be "portable" except in the coarsest sense.

Consider the complexities of limited resources: I'm short one Timer, but I figure out that on this specific chip I can have it feed into a CCP module and then use the latter's interrupt to effectively synthesize an extra Timer. How are you going to make that portable? Today's available timers are 8 bits, tomorrow's are 16 bits, different families have different numbers of 8 and 16 bit timers, some of the (E)CCP modules have unique features (example: timestamping inbound CAN messages), etc. This is why there are different libraries for different families, and why most skilled embedded developers don't bother with libraries anyway.

Again, I'm not trying to be a downer here. I love your enthusiasm and passion. But you openly stated "I've not devoted much time at all to language semantics yet". Semantics and implementation are almost the same thing in embedded environments. When you're dealing with a full blown, virtual memory, isolated execution context operating system you have the luxury of living in an academia-like "ideal world" where everything is fully abstracted. Embedded systems and microcontrollers aren't like that. Properly done, every line of embedded code is written while considering the blended hardware+firmware environment because every line can have ill effects.

I think this thread can have merit if you focus on asking people what problems they have, and then go off to consider how you'd solve those situations. So far the proposed "features" would be hindrences to me and I would aggressively avoid using them.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 07:14:37 pm
...

wouldn't have said it better.
Mentioning PICs, some of the proposed improvements are already part of the XC compilers, others are requests that clearly come from assembly programmers who can't stop thinking in assembly and ostentably write in what i call "Clever C" (forgetting that you CAN, and some time must, use separate assembly modules. Just conform to the ABI and call them as C functions so the compiler is happy, other programmers will be as well)
Dedicated strings requrie heap and personally speaking in a C project i prefer to avoid heap. In languages in which is a given i simply apply another mindset (using it responsibly of course)

I've only mentioned the kinds of things raised by very experienced hardware/software engineers that rely heavily on PIC, their needs, concerns and frustrations are real, not imagined, I am simply collating broad issues and concerns raised by people who find C restrictive or inflexible.

I do hope you aren't planning a language that is specific to current PIC processors and current PIC peripherals.

If that isn't the case, what will your strategy for inclusion/exclusion of a language feature? Either a small language that is all useful with all processors/peripherals. Or a large language that contains all possibilities, only some of which are exploitable with given processor/peripherals.

Without being able to articulate your strategy (to an audience including yourself), you will flail around in different directions.

These are good points. No, there is no special focus on PIC products, I just wanted to include some of the problems experienced in that domain, into the overall picture, if there was some very obscure thing, specific to some very narrow device then there's little to be gained from considering that unless the effort is not too high.

As for inclusion/exclusion strategies I think that's a fascinating question and I've not thought about it in any real detail yet.

It would include some kind of classification of language features I suppose. I suppose one could start listing features (whatever these get defined as) and then tabling  these for their utility across a range of devices.

In fact this is a very good question and I might take a stab at starting such a table with a view to having people more expert than I, contribute too and comment on.

I suppose we'd even need to define an MCU for the purpose of creating this tabulation.

At a first pass, these things might be inputs to how we do this:

(https://slideplayer.com/slide/13056543/79/images/8/Types+of+Microcontrollers.jpg)

and

(https://images.theengineeringprojects.com/image/main/2018/03/What-is-a-Microcontroller-Programming-Definition-Types-Examples-3.jpg)

Of course there are the peripherals to consider too.

(https://www.renesas.com/sites/default/files/inline-images/fig1-mcus-internal-conf-en.jpg)

Whether the language could or should abstract some aspects of some peripherals is an open question for me, ADC's are ADC's but there are sometimes fiddly, idiosyncrasies specific to some that are not seen in others, this is where my own knowledge is rather weak.

The bus width is a trait familiar to any language designer, so we expose numeric types that reflect that to some extent.

Interrupts are a clear thing to consider too, as are exceptions, also memory allocation, there are several ways to manipulate memory but most languages limit this to the concepts of static, stack and heap and offer limited way to interact with these, perhaps this is something that could be incorporated.

Anyway you do raise some important questions, clearly there is more to be done in this regard.



Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 07:17:15 pm
This is simply not true, I spoke recently with very experienced engineers who routinely use assembler for 8 bit PIC processors that are a large market. Trying to recode some parts of their existing, working designs in C is proving a huge challenge with code bloat and optimizations causing serious problems. Simple optimizations too like short-circuit evaluations.
Code bloat is pretty much inevitable when using a high level language for a PIC or an 8051. There was one very focused C compiler for tiny cores. MicroChip bought it, and shut down things like the 8051 version, which had been quite popular for putting fairly complex apps on 8051 cores. That left the market with no dense code compilers for the 8051, and a reasonably dense compiler for the PIC. If you think code bloat is bad with the PIC compiler, you should try looking at some 8051 apps.

True, I wonder what underlies this "bloat"? experienced assembly language programmers probably have great insights here and if there is some "bloat" due to a language itself rather than poor code generation and optimization, it would be interesting to quantitatively, somehow, measure this.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 07:19:16 pm
This is simply not true, I spoke recently with very experienced engineers who routinely use assembler for 8 bit PIC processors that are a large market. Trying to recode some parts of their existing, working designs in C is proving a huge challenge with code bloat and optimizations causing serious problems.

(emphasis mine.) Again being rude (to other people, too:) sounds like peter-h is one of your very experienced engineers.

I can relate, I have written cycle-accurate code on 8-bitters and so on, but got into more challenging projects more than 15 years ago. I still believe what you propose is more like a niche tool than actually useful as a generic paradigm, or a necessity.

Old projects, on old HW,  written using old-time methods - let them be, or if you absolutely must change them, find people who can work like "back then". If large changes are needed, redesign using today's HW and design principles.

I'm suspicious about the need of doing large, fundamental rewrites to 8-bit projects without hardware changes, and I'm also suspicious about the supposedly large size of these 8-bit projects. They are usually quite small, given the limitations in memory.

Squeezing out the very last drops of memory or CPU performance (micro-optimization) is specialized work anyway. If you are going to sell 10000 units you just use large enough MCU and pay some dozen cents more for it and don't need to optimize until cows come home. If you are going to sell 10000000 units, then it makes financial sense to squeeze hell out of it to save $0.10 * 10000000 = $1000000, but then you can afford a real professional who does the job in a few days and does not complain about needing a new language. Been there, done that, too.

I recently came across this in my studies:

https://www.youtube.com/watch?v=exOwctrlB7o (https://www.youtube.com/watch?v=exOwctrlB7o)

Now I interpret that to mean that 8 bit devices are alive and well and very much in demand.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 07:26:59 pm
C is not and never was designed for MCU programming
True, and yet it's just about the perfect "universal language" for embedded applications. It's close to the hardware without (necessarily) requiring the user to learn the specifics of the chip. Bloat is often related to a language's abstraction level, after all. Just as bloat is often related to the programmer's inexperience with the underlying hardware.

As for a "language that is [processor] specific", that's what libraries are for. And that's not limited to C, either. All the common, basic constructs are part of the language whether it's C, C++, Python, Basic, etc. Then, if you don't want to roll your own interface to the specific hardware, you grab a library. Sure it will be bloated since libraries generally try to be all things to all people, but that's the price you pay for the convenience of being abstracted (there's that word again!) from the hardware details.

I suspect all you're going to get with an "embedded language" is something that looks like C with a bunch of libraries - and that exists today. And it will be specific to that one family... so you'll have different "language versions" for various processors. Even the API won't be the same because the on-chip peripherals differ from one family to another, so code won't be "portable" except in the coarsest sense.

Consider the complexities of limited resources: I'm short one Timer, but I figure out that on this specific chip I can have it feed into a CCP module and then use the latter's interrupt to effectively synthesize an extra Timer. How are you going to make that portable? Today's available timers are 8 bits, tomorrow's are 16 bits, different families have different numbers of 8 and 16 bit timers, some of the (E)CCP modules have unique features (example: timestamping inbound CAN messages), etc. This is why there are different libraries for different families, and why most skilled embedded developers don't bother with libraries anyway.

Again, I'm not trying to be a downer here. I love your enthusiasm and passion. But you openly stated "I've not devoted much time at all to language semantics yet". Semantics and implementation are almost the same thing in embedded environments. When you're dealing with a full blown, virtual memory, isolated execution context operating system you have the luxury of living in an academia-like "ideal world" where everything is fully abstracted. Embedded systems and microcontrollers aren't like that. Properly done, every line of embedded code is written while considering the blended hardware+firmware environment because every line can have ill effects.

I think this thread can have merit if you focus on asking people what problems they have, and then go off to consider how you'd solve those situations. So far the proposed "features" would be hindrences to me and I would aggressively avoid using them.

Again, this raises excellent points, I will re-read your post again later. But did want to ask you which of these would hinder you rather than help:



What are the kinds of things you'd personally include in a list like this?

Consider the namespaces question, I implemented this in C recently (https://github.com/Steadsoft/embedded/blob/main/NucleoF446RE/NRF24/Transmitter/nrf_hal_tx_project.c#L130) as part of a small hobby project, it would have been so much easier and lower maintenance and higher performance if the language just exposed a "namespace" keyword or something.

What about a computed goto:

Code: [Select]

handler = calculate_handler();

goto manage_event[handler];


We can have an array of function pointers in C but we can't have any arrays of labels, goto targets, again this is low cost from a language design standpoint (as is namespaces) yet a pretty helpful thing. Using function pointers works too, can be coded but then if we had some nested design, where blocks of code goto blocks of code that might also goto blocks of code...the overhead of call/return/push/pop starts to grow yet vanishes with a computed goto.

The computed goto could, might, prove helpful in the implementation of state machines without the cost incurred by using tables of function pointers, especially in designs where the handlers to very little and the cost of invoking them exceeds the cost of the work they might do.


Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 23, 2022, 07:48:52 pm
I personally think your list is a mixed bag with a lot of too specific features (such as the computed gotos and the cycle-based timing) which seem odd at best, but that's just an opinion.
For a lot of the rest, this is actually available in... Ada. Problem sure is that Ada tends to yield large object code so not really adapted to small targets.

As to specifically namespaces, I agree, but I do think the "module" approach is much, much better than raw namespaces. Namespaces are a hack.

I've been saying that C lacks modules, for a long time. They introduced modules in C++ recently, but the definition they introduced is IMO a joke. Proper modules? Just look at Modula-2, Modula-3, Oberon, Ada packages. I particularly like Modula-3 modules that can be parameterized, Ada packages as well.

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 07:54:09 pm
I personally think your list is a mixed bag with a lot of too specific features (such as the computed gotos and the cycle-based timing) which seem odd at best, but that's just an opinion.
For a lot of the rest, this is actually available in... Ada. Problem sure is that Ada tends to yield large object code so not really adapted to small targets.

As to specifically namespaces, I agree, but I do think the "module" approach is much, much better than raw namespaces. Namespaces are a hack.

I've been saying that C lacks modules, for a long time. They introduced modules in C++ recently, but the definition they introduced is IMO a joke. Proper modules? Just look at Modula-2, Modula-3, Oberon, Ada packages. I particularly like Modula-3 modules that can be parameterized, Ada packages as well.

Tell me about "modules" how do they work? what do they buy you? I'm interested in hearing this from a real world "language user" so to speak! Are they for binary reuse? or ways to package source code for easier reuse?
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 08:00:04 pm
I wanted something like this "module" idea and became fixated on it recently, in fact this was a trigger for me getting interested in the whole language thing.

I crafted a way of working that would be much nicer if innately supported at a language level, you can read what I did here (https://github.com/Steadsoft/embedded/tree/main/NucleoF446RE/NRF24/Transmitter), I documented it for my own sake and to sanity check myself.
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 23, 2022, 08:09:05 pm
This is simply not true, I spoke recently with very experienced engineers who routinely use assembler for 8 bit PIC processors that are a large market. Trying to recode some parts of their existing, working designs in C is proving a huge challenge with code bloat and optimizations causing serious problems. Simple optimizations too like short-circuit evaluations.

That is because 8 bit PIC is utterly unsuited to running any kind of modern high level compiled language.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 08:30:09 pm
This is simply not true, I spoke recently with very experienced engineers who routinely use assembler for 8 bit PIC processors that are a large market. Trying to recode some parts of their existing, working designs in C is proving a huge challenge with code bloat and optimizations causing serious problems. Simple optimizations too like short-circuit evaluations.

That is because 8 bit PIC is utterly unsuited to running any kind of modern high level compiled language.

That can't be true Bruce, CP/M for example - an OS no less - was written in PL/I a high level compiled language, CP/M ran on the 8080, Z80 and 8085 all 8 bit devices and has been ported to other 8 bit chips too.

The device doesn't "run a language" either, it runs machine code, so long as that code has been produced sensibly for the target it will run no differently to any code surely?

Also C cannot be described as a "modern language" by any stretch, it must be fifty years old.




Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 23, 2022, 09:16:17 pm
This is simply not true, I spoke recently with very experienced engineers who routinely use assembler for 8 bit PIC processors that are a large market. Trying to recode some parts of their existing, working designs in C is proving a huge challenge with code bloat and optimizations causing serious problems. Simple optimizations too like short-circuit evaluations.

That is because 8 bit PIC is utterly unsuited to running any kind of modern high level compiled language.

That can't be true Bruce, CP/M for example - an OS no less - was written in PL/I a high level compiled language, CP/M ran on the 8080, Z80 and 8085 all 8 bit devices and has been ported to other 8 bit chips too.

And this has what to do with PIC, exactly?

8080/z80 is about 100x more compiler-friendly than PIC. But about 10x less compiler-friendly than the also 8 bit 6809 or AVR.

Quote
The device doesn't "run a language" either, it runs machine code, so long as that code has been produced sensibly for the target it will run no differently to any code surely?

You're talking to someone who writes compilers, and also designs new CPU instructions. Thanks for the lesson.

8 bit PIC is fundamentally unsuited to at least the following features of modern programming languages:

- pointers and anything that is not located at an absolute address

- constant data structures in ROM

- recursive or reentrant functions

- deep call chains at all

- runtime code generation

Quote
Also C cannot be described as a "modern language" by any stretch, it must be fifty years old.

Ahead of its time, clearly.

Sadly, it is becoming more and more apparent that you don't know the first thing about either CPU instruction sets or programming languages, and you won't listen to those who do, so I'm out of this conversation.
Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 23, 2022, 09:16:56 pm
I've used 8-bit PICs in the past, and while for the 16F series I only used assembly (had tried a C compiler back then, but it was just not worth it), for the 18F series, I only used C. The mcc18 compiler produced decent code for the 18F. Never had any issue with that. I've also used SDCC on 8051 stuff (Cypress FX1/FX2) with success. So, certainly usable.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 09:47:04 pm
This is simply not true, I spoke recently with very experienced engineers who routinely use assembler for 8 bit PIC processors that are a large market. Trying to recode some parts of their existing, working designs in C is proving a huge challenge with code bloat and optimizations causing serious problems. Simple optimizations too like short-circuit evaluations.

That is because 8 bit PIC is utterly unsuited to running any kind of modern high level compiled language.

That can't be true Bruce, CP/M for example - an OS no less - was written in PL/I a high level compiled language, CP/M ran on the 8080, Z80 and 8085 all 8 bit devices and has been ported to other 8 bit chips too.

And this has what to do with PIC, exactly?

8080/z80 is about 100x more compiler-friendly than PIC. But about 10x less compiler-friendly than the also 8 bit 6809 or AVR.

Quote
The device doesn't "run a language" either, it runs machine code, so long as that code has been produced sensibly for the target it will run no differently to any code surely?

You're talking to someone who writes compilers, and also designs new CPU instructions. Thanks for the lesson.

8 bit PIC is fundamentally unsuited to at least the following features of modern programming languages:

- pointers and anything that is not located at an absolute address

- constant data structures in ROM

- recursive or reentrant functions

- deep call chains at all

- runtime code generation

Quote
Also C cannot be described as a "modern language" by any stretch, it must be fifty years old.

Ahead of its time, clearly.

Sadly, it is becoming more and more apparent that you don't know the first thing about either CPU instruction sets or programming languages, and you won't listen to those who do, so I'm out of this conversation.

Please review our conversation. I described an engineer who was encountering significant difficulties writing some of his existing (large) assembler code into C. The C he was using is supplied by Microchip for their 8 bit PIC devices.

So before making sweeping pronouncements like "8 bit PIC is utterly unsuited to running any kind of modern high level compiled language" you need to ask yourself why Microchip are seemingly unaware of this.

That you happen to know more than I about PIC does not justify insults either, whatever happened to mature, mutually respectful, polite discourse...

Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 23, 2022, 10:16:14 pm
So before making sweeping pronouncements like "8 bit PIC is utterly unsuited to running any kind of modern high level compiled language" you need to ask yourself why Microchip are seemingly unaware of this.

I can assure you, Microchip is painfully well aware of this.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 23, 2022, 10:33:28 pm
So before making sweeping pronouncements like "8 bit PIC is utterly unsuited to running any kind of modern high level compiled language" you need to ask yourself why Microchip are seemingly unaware of this.

I can assure you, Microchip is painfully well aware of this.

Would you say then that this could also apply to them "it is becoming more and more apparent that they don't know the first thing about either CPU instruction sets or programming languages, and they won't listen to those who do".

Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 23, 2022, 11:02:32 pm
So before making sweeping pronouncements like "8 bit PIC is utterly unsuited to running any kind of modern high level compiled language" you need to ask yourself why Microchip are seemingly unaware of this.

I can assure you, Microchip is painfully well aware of this.

Would you say then that this could also apply to them "it is becoming more and more apparent that they don't know the first thing about either CPU instruction sets or programming languages, and they won't listen to those who do".

No, because they've been trying for decades to move people to their (licensed-in or outright bought) AVR, PIC32, and ARM product lines, all of which support high level languages such as C very well. Possibly dsPIC too, but I don't know anything about it.
Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 23, 2022, 11:19:44 pm
Would you say then that this could also apply to them "it is becoming more and more apparent that they don't know the first thing about either CPU instruction sets or programming languages, and they won't listen to those who do".
There is only so much you can do to improve architecture from 1976. PICs are really bad (by modern standard) as far as architecture goes. But people keep buying them and they have the highest profit margin. There is no incentive to do anything here.

Smart people will buy much cheaper an more capable ARM-based MCUs. But there are a lot of people stuck in the past and afraid of change.

There are also legacy designs that don't need a lot of maintenance anymore.
Title: Re: A new, hardware "oriented" programming language
Post by: rstofer on November 23, 2022, 11:53:04 pm
I spent quite a bit of time with the PIC 16F877(A) and I have the following observations:

The stack is only 8 addresses deep and there are no push or pop instructions to allow it to be used for passing function arguments.  C calling conventions are right out the window.  A stack has to be created and managed in software.

Emitted code from C compilers will spend a lot of space on code for bank and page switching.  There's only 8k of flash and the C compilers I have used burned through a lot of it.  I haven't looked at the PIC 16F series in about 10 years, maybe the C compilers are better but I doubt it.  The hardware simply doesn't support high level languages.

There are only 36 instructions and assembly language programming is fairly simple.  There simply are no complex addressing schemes and everything is dead simple.

Code will be tight, it will be laid out to eliminate paging as much as possible and bank switching will also be optimized.

The fact that the mid-range PIC is still around is testament to the fact that many applications can run on processors with just a single core and no hardware support for higher level languages.

I prefer some of the Atmel chips like the ATmega128.  It is a great chip to program with C.  The Arduino AMmega328P chip is also fun to work with at the bare metal level.


Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 23, 2022, 11:58:51 pm
I do hope you aren't planning a language that is specific to current PIC processors and current PIC peripherals.

If that isn't the case, what will your strategy for inclusion/exclusion of a language feature? Either a small language that is all useful with all processors/peripherals. Or a large language that contains all possibilities, only some of which are exploitable with given processor/peripherals.

Without being able to articulate your strategy (to an audience including yourself), you will flail around in different directions.

These are good points. No, there is no special focus on PIC products, I just wanted to include some of the problems experienced in that domain, into the overall picture, if there was some very obscure thing, specific to some very narrow device then there's little to be gained from considering that unless the effort is not too high.

As for inclusion/exclusion strategies I think that's a fascinating question and I've not thought about it in any real detail yet.

It would include some kind of classification of language features I suppose. I suppose one could start listing features (whatever these get defined as) and then tabling  these for their utility across a range of devices.

Yes, it is an interesting question - and one that you can see being played out in all sorts of arenas, for a variety of reasons. Some reasons are academic, some commercial, some pragmatic, some concealed.

There's no single correct answer. But the wrong  answer often starts with not stating clearly the objectives and non-objectives.

Quote
Anyway you do raise some important questions, clearly there is more to be done in this regard.

Frequently answering questions is trivial, but knowing the right question to ask is more difficult and more important.

Over the decades I've come to appreciate skill in asking the right simple question - one  where the answer illuminates all sorts of assumptions and biasses.

For hardware, "what are the various clock domains?"

For software, "what is the concept and definition of 'address'?" What is the definition of "unique"?
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 24, 2022, 12:03:42 am
This is simply not true, I spoke recently with very experienced engineers who routinely use assembler for 8 bit PIC processors that are a large market. Trying to recode some parts of their existing, working designs in C is proving a huge challenge with code bloat and optimizations causing serious problems. Simple optimizations too like short-circuit evaluations.

That is because 8 bit PIC is utterly unsuited to running any kind of modern high level compiled language.

That can't be true Bruce, CP/M for example - an OS no less - was written in PL/I a high level compiled language, CP/M ran on the 8080, Z80 and 8085 all 8 bit devices and has been ported to other 8 bit chips too.

And this has what to do with PIC, exactly?

8080/z80 is about 100x more compiler-friendly than PIC. But about 10x less compiler-friendly than the also 8 bit 6809 or AVR.

Spot on!

(Except that the Z80 has far too many gaping holes in its instruction map!)
Title: Re: A new, hardware "oriented" programming language
Post by: nctnico on November 24, 2022, 12:08:52 am
Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.
That one is certainly not going to happen. Very few machines now permit cycle exact execution of anything. Even really small devices have at least some elementary kind of cache, that messes with the instruction timing.
That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C. There are assembler MCU developers out there that struggle to use C because of this kind of thing. They have carefully crafted code where the want some execution path to take exactly the same number of clock cycles as some other, so they embed multiple NOPs sometimes, their designs require that.
The days that you need exact execution times are long gone. Any language chasing features like that is outdated before it has been designed.

The future is with languages that are better at dealing with complex firmware and take away pitfalls like pointers and buffer overruns from programmers so that effort can be directed at adding functionality.

Languages like Rust, Lua and Python make much more sense on a microcontroller nowadays.
Title: Re: A new, hardware "oriented" programming language
Post by: rstofer on November 24, 2022, 12:28:49 am
There is one application where exact loop timing is required: Directly generating graphic images for games.  Typically the superloop will have several paths depending on which layer of the image it is generating.  Sprites come to mind.  It is critical that every pixel lay down at exactly the right time if the image is to make sense.  This kind of thing was common on the Apple II and the Commodore 128.

Clearly, it is an old concept but it worked real well as long as the cycle count was accurate..

Look at line 276+ for display loop timing.  There is also a specific delay() that is used throughout the program.

Grab the .zip file at the bottom of the page
https://www.dos4ever.com/upong/upong.html (https://www.dos4ever.com/upong/upong.html)

This is a PAL implementation but there are NTSC versions out on the web.
Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 24, 2022, 12:36:55 am
Who is generating graphics on the fly from the microcontroller anymore?
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 24, 2022, 12:50:12 am
There is one application where exact loop timing is required: Directly generating graphic images for games.  Typically the superloop will have several paths depending on which layer of the image it is generating.  Sprites come to mind.  It is critical that every pixel lay down at exactly the right time if the image is to make sense.  This kind of thing was common on the Apple II and the Commodore 128.

Clearly, it is an old concept but it worked real well as long as the cycle count was accurate..

VGA 640x480@30 dot rate is 12.58 MHz. Or 1.57 MHz for each byte of b&w pixels.

Sure, that's tough on a 1 or 2 MHz machine.

But do it on a 50+ MHz machine and you can afford to take an interrupt and save all your registers for EVERY BYTE. With 500 MHz you could take a timer interrupt for every pixel.
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 24, 2022, 12:51:13 am
Who is generating graphics on the fly from the microcontroller anymore?

Atari 2600 chads. Gigatron.
Title: Re: A new, hardware "oriented" programming language
Post by: rstofer on November 24, 2022, 01:11:27 am
Retro is a big deal! 

I have implemented several CP/M machines over the last dozen years.  I still enjoy using it and, at 50 MHz, it scoots right along.

Then there is the FPGA implementation of the IBM1130 that I first started programming in 1970 (FORTRAN and a bit of RPG). Certainly retro because the original machine ran at around 400 kHz and my FPGA runs at 50 MHz.  I swear I'm going to move it to an Artix 7 and kick it up to 100 MHz.  One of these days...

Then there is the FPGA implementation of a Z80 making up PacMan - more retro!  We're talking PacMan with the original ROMs (converted to BlockRAM of course).  Very cool!

Yes, I have the Teensy 4.1 when I want to get things done in a hurry.  I like the LPC1768 mbed but it is now obsolete and ARM is changing the toolchain.  I need to do some testing...

Pong on an 8 pin uC?  Most people think 8 pins is relegated to 555 projects or op amps...  Very retro.

Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 24, 2022, 01:19:03 am
Even if you are, *with proper peripherals* you absolutely do not need to have code execute with exact timings - you only need it to execute within some max time.
Only if you use old MCUs with basically no peripheral other than very basic stuff and very limited RAM (thus preventing any double buffering) do you have to time code like this. Or even older CPUs.

Coding like this on any modern MCU doesn't make sense, and I think that was the question ataradov asked. Now emulating old gear with modern hardware is a weird endeavour with very much its own weird "rules". So, anything goes with a rationale that only people doing it can explain to themselves.

In other words, can you code emulators even on modest modern MCUs without coding like this, having to time code execution to the cycle? Absolutely. Are there still people willing to code to the cycle using old methods? Absolutely too. To each their own.
Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 24, 2022, 01:22:37 am
Atari 2600 chads. Gigatron.
Ok, but that's not a typical use case. This is just doing random stuff for fun. Designing the language that would align stuff for you just for this is an overkill.

And people working on Gigatron managed to get things going without that.

I can't imagine any actual mass market application for that.
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 24, 2022, 01:24:20 am
Pong on an 8 pin uC?  Most people think 8 pins is relegated to 555 projects or op amps...  Very retro.

8 pins? GND and Vcc, two ADCs for paddles, start/pause button, video out.

10 cent 8 pin 48 MHz 32 bit RISC-V with 16 KB flash and 2 KB RAM? Might just about manage Pong :-)

With 16-ε registers, you can probably do it with zero RAM, just the MMIO.
Title: Re: A new, hardware "oriented" programming language
Post by: lyxmoo on November 24, 2022, 01:54:16 am
less code, or minimize code under MCU enviroment, it's will be great for no code with programming just like PLC, combine functional bricks together.
Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 24, 2022, 01:59:00 am
Different applications have different needs. Programming PLCs is like hell on Earth for me personally. I would not want to program general purpose stuff that way.

Plus there are drag and drop systems. They are just not very useful for real use. Plus doing "code" versioning on graphics input stuff is next to impossible. This would throw away all the modern development flows.
Title: Re: A new, hardware "oriented" programming language
Post by: JPortici on November 24, 2022, 05:23:36 am
That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C. There are assembler MCU developers out there that struggle to use C because of this kind of thing. They have carefully crafted code where the want some execution path to take exactly the same number of clock cycles as some other, so they embed multiple NOPs sometimes, their designs require that.

Simple enough microcontrollers that can produce cycle-accurate predictable timing by simple instruction counting are becoming exceedingly rare. These 8-bitters still exist, but such cycle-accuracy combined to generally very crappy performance is something people do not actually want. Instead, people buy modern high-performance microcontrollers which can still produce very accurate timing by just scaling the absolute time used per cycle down, by utilizing higher clock speed. For example, a 12-cycle ISR latency on a 400MHz MCU looks like HALF a cycle on a 16MHz AVR/PIC; at that point you simply do no care if it sometimes takes 14 cycles due to some pipeline or branch mispredict or something. I have done pretty timing-sensitive things simply on interrupt handlers, and the advantage is ease of writing, reading, and maintaining that code. Manual cycle counting, or automated version thereof, is simply not needed anymore, except in very rare cases, which require careful understanding anyway.

The problem with this "predictive timing language" is, it becomes tied to certain type of hardware, and then you have replicated what the XCORE folks have done (apparently pretty well).

This is simply not true, I spoke recently with very experienced engineers who routinely use assembler for 8 bit PIC processors that are a large market. Trying to recode some parts of their existing, working designs in C is proving a huge challenge with code bloat and optimizations causing serious problems. Simple optimizations too like short-circuit evaluations.

All I'm arguing here really is for a new language to have features that help avoid these kinds of problems, where some aspects of the generated code are more controllable in a fine grained way, the NOP idea is just an example of what's required rather than a formal proposed language feature.

I believe you.
What i do in these cases is write an assembly module for the really time critical stuff (or stuff that doesn't translate well in c) and link that in, call it like a C function if it doesn't live and die on its own context (ISR).
Bulk of the firmware stays in C with all the benefits of moving from assembly, as only the truly special parts are in assembly.
Granted it's more difficult for 8bit PIC if you use XC8/HiTech C8 because of the compiled stack, but it's not impossible.
That doesn't make me feel the need for a special language or a special compiler for which i have to remember the dialect

Also, bit manipulation, exact number of NOPs, all things that can be done in XC8 by the use of the compiler builtins. Maybe it won't solve every issue, but it's there
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 24, 2022, 10:27:10 am
Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.
That one is certainly not going to happen. Very few machines now permit cycle exact execution of anything. Even really small devices have at least some elementary kind of cache, that messes with the instruction timing.
That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C. There are assembler MCU developers out there that struggle to use C because of this kind of thing. They have carefully crafted code where the want some execution path to take exactly the same number of clock cycles as some other, so they embed multiple NOPs sometimes, their designs require that.
The days that you need exact execution times are long gone. Any language chasing features like that is outdated before it has been designed.

True, but... in a hard realtime application you do need to know the worst case loop execution time and/or latency between input and output.

That implies accurate cycle counting, but not NOP insertion.


Quote
The future is with languages that are better at dealing with complex firmware and take away pitfalls like pointers and buffer overruns from programmers so that effort can be directed at adding functionality.

Languages like Rust, Lua and Python make much more sense on a microcontroller nowadays.

Python has the global interpreter lock and can garbage collect. If those aren't a problem for an application, fine.

In the medium term I would bet on Rust becoming important for embedded systems. At the moment it is on the bleeding edge.

I don't know enough about Lua to have a valid opinion.

Overall the language is relatively unimportant, provided it can control the hardware as expected and you can get staff willing to take the time to have it on their CV. The latter is a significant hurdle for custom languages, including DSLs.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 24, 2022, 10:29:54 am
Even if you are, *with proper peripherals* you absolutely do not need to have code execute with exact timings - you only need it to execute within some max time.

That's the point!

With modern MCUs becoming ever more powerful (good), complex (bad), and complicated (bad bad), knowing the worst case timing is becoming more difficult. In a hard realtime environment, the mean/typical timing is irrelevant.
Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 24, 2022, 03:27:02 pm
With modern MCUs becoming ever more powerful (good), complex (bad), and complicated (bad bad), knowing the worst case timing is becoming more difficult. In a hard realtime environment, the mean/typical timing is irrelevant.
I believe, that –– perhaps surprisingly! –– distributed processing is a valid solution to this.

In a very real sense, it reduces complications, because tasks are done in separate, isolated units, that simply communicate with each other.
Message passing interfaces (with input buffers, mailboxes, and/or DMA and completion signals/flags) lets computation proceed simultaneously with communication, but it is much easier to provide/prove worst case timings for any computation.

For example, instead of integrating quadrature encoders in an MCU, I'd prefer to stick a few-cent dedicated chip on each encoder, with a shared bus to the MCU, with the MCU just accumulating the encoder state changes in a dedicated "register" per encoder.  The MCU hardware peripheral is a bit more complex, but would need few pins to support even a large number of encoders.  No software interrupts would be required for normal operation, so software timing would not be affected.  (I'd use two addresses for each "register", though: one for normal reads, and the other for reads that clear the "register" atomically when read, for easy delta tracking without having to worry about atomicity.)

Then again, I might just be crazy.  :-//
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 24, 2022, 03:41:06 pm
With modern MCUs becoming ever more powerful (good), complex (bad), and complicated (bad bad), knowing the worst case timing is becoming more difficult. In a hard realtime environment, the mean/typical timing is irrelevant.
I believe, that –– perhaps surprisingly! –– distributed processing is a valid solution to this.

In a very real sense, it reduces complications, because tasks are done in separate, isolated units, that simply communicate with each other.
Message passing interfaces (with input buffers, mailboxes, and/or DMA and completion signals/flags) lets computation proceed simultaneously with communication, but it is much easier to provide/prove worst case timings for any computation.

For example, instead of integrating quadrature encoders in an MCU, I'd prefer to stick a few-cent dedicated chip on each encoder, with a shared bus to the MCU, with the MCU just accumulating the encoder state changes in a dedicated "register" per encoder.  The MCU hardware peripheral is a bit more complex, but would need few pins to support even a large number of encoders.  No software interrupts would be required for normal operation, so software timing would not be affected.  (I'd use two addresses for each "register", though: one for normal reads, and the other for reads that clear the "register" atomically when read, for easy delta tracking without having to worry about atomicity.)

Then again, I might just be crazy.  :-//

No, you are not crazy. That is an extremely sane response.

It even exists for hard realtime embedded systems, and you can buy it from DigiKey:

Further advances will need to be made in the heavily multicore future, but xCORE+xC shows what can already easily be achieved in this space. Shame most of the world is left sticking band aid plasters on environments that were state of the art half a century ago.

We'll get to A Better Place (just as COBOL has been superceded), but it won't be in my lifetime.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 04:26:05 pm
Well, some very interesting posts recently, I appreciate this kind of discussion.

I'm now satisfied that the PIC 8 stuff is a bit of an odd man out, the exception rather than the typical.

This wasn't apparent earlier, it does seem like it is pretty unusual architecturally.

As for the precise timing stuff though, this strikes me as being a specific kind of optimization, rather than a language feature. I mean the language could expose some keyword or something to identify such code, but ultimately it becomes just a specific kind of optimization.

So if incorporating this isn't a huge cost then I lean toward including this a goal, at least until some good reason comes up to abandon the idea.

The motive for this and some other things too, is to free the mind from preconceptions, use a clean sheet of paper so to speak and not subconsciously constrain a new language.

Even features that might only be used by a small number of users are sometimes still desirable to have if the cost of including them isn't high.

As for Rust that is a "better" language that C but I learned recently from someone here of something even better Zig. and here's a very detailed blog post (https://kevinlynagh.com/rust-zig/) by a clearly competent professional, as to why he is impressed by Zig.

He was impressed by the "inline for" feature too, and the language exposes "vectors" these are the kinds if things that interest me.

I'd be interested to hear what things about Zig do not appeal to experienced MCU developers, it looks like Zig is rather good, but it perhaps was not designed with MCU's in mind so might have gaps in that respect, note too that a motivation for Zig was to improve upon C.

One advocate of Zig points this out, a view I share wholeheartedly:

Quote
"The Zig project for me," Andrew tells us, "was peeling off these abstraction layers [of C] that we've all taken for granted and examining their foundational premises and saying, 'Is that really how it has to work? Let's question some of this stuff. Maybe it can be better'."

While investigating why features in C operate the way they do, Andrew came to realize that the reasoning behind many of them are not well justified.






Title: Re: A new, hardware "oriented" programming language
Post by: alexanderbrevig on November 24, 2022, 05:12:34 pm
The best programming language for embedded already exists!
Rust is amazing an a very good fit.

The notion that C or C++ is better is simply wrong.
It's better both in term of what people prefer (stack overflow) and by the number of bugs that pass by the compiler.
Stay relevant tomorrow and learn it today :)  8)
Title: Re: A new, hardware "oriented" programming language
Post by: newbrain on November 24, 2022, 05:26:56 pm
Stay relevant tomorrow and learn it today :)  8)
You know what?
I'm intrigued.
I have a small project coming: a simple SWR meter, based on the AD8310 and an ADC, a Pico board (or whatever I have at hand) and an I2C OLED display.

The HW is quite trivial, the SW too, I think I can do away without user input, so I think I'll try to go Rust.
We'll see how it pans out.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 05:27:49 pm
Another thing to look at with a language is the way we interact with heaps and allocate and free and so on. I've written rather sophisticated concurrent shared memory allocators before and I have insights into this area.

Something that might help in an MCU world is not only different heap strategies but also dynamically created heaps, heaps created for use in certain bits of code for example.

Also heap metrics might be useful, the ability for code to dynamically probe a heap for total memory use and so on or gross and net allocation counts and so on.

These could be the province of a library of course but there might be some value in doing some of this within a language itself.

Looking at PL/I again we see it support several interesting ideas with memory, one is "controlled storage" effectively a stack concept. One can allocate a variable and then allocate it again and then free it, and the variable then refers to the first allocation. This isn't done through pointers, all the developer sees is a variable.

In pseudo code:

Code: [Select]

dcl measurement_details SomeType controlled;

allocate measurement_details;

// do stuff with the variable measurement_details.

...

// allocate another.

allocate measurement_details;

free measurement_details;

// adjust the initially allocated measurement_details
...

measurement_details.totals = 10;


Then there is also "based" declarations where the address of some datum is specified at declaration time and used implicitly thereafter.

Some of these memory specific features might have utility in an MCU world.

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 05:28:59 pm
The best programming language for embedded already exists!
Rust is amazing an a very good fit.

The notion that C or C++ is better is simply wrong.
It's better both in term of what people prefer (stack overflow) and by the number of bugs that pass by the compiler.
Stay relevant tomorrow and learn it today :)  8)

Rust is indeed popular but I learned recently of Zig and it seems it has much to offer the MCU developer.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 24, 2022, 05:51:54 pm
The best programming language for embedded already exists!
Rust is amazing an a very good fit.

The notion that C or C++ is better is simply wrong.
It's better both in term of what people prefer (stack overflow) and by the number of bugs that pass by the compiler.
Stay relevant tomorrow and learn it today :)  8)

Rust is indeed popular but I learned recently of Zig and it seems it has much to offer the MCU developer.

I had a quick scan of the Zig blog you referred to, and I wasn't impressed. To over-simplify, it seemed to be too hung up on a few bits of syntax, and didn't address the semantics of the languages or the appropriate use-cases for languages.

Fundamentally syntax isn't very interesting: I can learn to type different things to represent the same concepts - but I will only do so if the language semantics give me some real advantages over existing languages. Hence I avoided learning Delphi because it didn't offer much over C. I ignored C# since it offers nothing over Java (and introduces significant problems too). Etc etc.

To over-simplify again, semantics is about "what don't I have to worry about because I know the language will correctly deal with it on my behalf as my programs get bigger and more complicated". One such example is memory management. Another is parallel processing, especially with many cores/processors. Another is "will my code meet the hard realtime deadline". And, of course, the interactions between all those.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 06:30:29 pm
The best programming language for embedded already exists!
Rust is amazing an a very good fit.

The notion that C or C++ is better is simply wrong.
It's better both in term of what people prefer (stack overflow) and by the number of bugs that pass by the compiler.
Stay relevant tomorrow and learn it today :)  8)

Rust is indeed popular but I learned recently of Zig and it seems it has much to offer the MCU developer.

I had a quick scan of the Zig blog you referred to, and I wasn't impressed. To over-simplify, it seemed to be too hung up on a few bits of syntax, and didn't address the semantics of the languages or the appropriate use-cases for languages.

Fundamentally syntax isn't very interesting: I can learn to type different things to represent the same concepts - but I will only do so if the language semantics give me some real advantages over existing languages. Hence I avoided learning Delphi because it didn't offer much over C. I ignored C# since it offers nothing over Java (and introduces significant problems too). Etc etc.

To over-simplify again, semantics is about "what don't I have to worry about because I know the language will correctly deal with it on my behalf as my programs get bigger and more complicated". One such example is memory management. Another is parallel processing, especially with many cores/processors. Another is "will my code meet the hard realtime deadline". And, of course, the interactions between all those.

Well that's fine, at least you've looked at it, its a rich subject and a huge source of healthy disagreement!

Speaking of disagreement, you say "I ignored C# since it offers nothing over Java (and introduces significant problems too)".

I'd be interested in hearing more on the problems you found.

I use C# professionally and have used it since the earliest beta, I also often participate in the language discussion on Github where language changes and enhancements are hatched.

I'm not aware any linguistic advantage Java has over C#, Java is not standardized either whereas C# is defined by an international, non proprietary standard.

Java (because it is based on the C grammar) find its very hard to add new keywords and grammar changes, for example Java does not have properties, so one has to code get_ and set_ or some other contrivance.

Java does not have generics, in C# generics are instantiated at runtime, Java has a limited compile-time form of generics (type erasure) with consequent serious limitations.

C# has partial classes greatly simplifying system that rely in custom generated source code.

C# has indexers.

C# has ref/out argument passing and supports optional arguments and positional arguments, not so Java.

C# has superb interop capabilities Java has very little to offer.

C# offers pointers for scenarios where code must interact with memory not managed by the CLR.

C# supports iterators, async iterators, async and await as language concepts.

C# has come a long way over the past twenty years, my only criticism of C# is that it might be getting a bit unwieldy and some of that is due to (IMHO) the fact that it too, presumed the C grammar rather than a fresh clean sheet of paper approach.

Also you say "syntax isn't very interesting" and I don't know what that means, do you mean you don't care at all about notational aspects of a language? We reason about code by looking at it, a good syntax improves one's ability to reason over a poor syntax.

This is why mathematics succeeds, it us superb when it comes to notation, we can look at some mathematical equation and reason about it, but express the same equation in assembler or C and we lose that ability to a great extent.

The king of languages from the standpoint of notation is APL, initially that was an experiment in a new mathematical notation for certain types of problems, later it became a programming language and remains to this day the pinnacle of expressivity.

Iverson - who invented APL - wrote a superb paper on this, it was a talk he gave after receiving the Turing Award for his contribution to computer science, syntax is everything to Iverson.

Notation as a tool of thought (https://www.jsoftware.com/papers/tot.htm).

In a similar way lambda calculus is better at expressing certain ideas than a Turing machine is.














Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 24, 2022, 06:39:19 pm
The prevalence of buffer overflow/underflow/overrun bugs in C is at least partly explained in that the fundamental unit of reference in the libraries is an address instead of an address range.

If we look at languages that natively support slicing, they conceptually use tuples (origin, step, length) or (origin, step, min-index, max-index) to describe array slices.

While current C compilers can check for array boundaries at compile time (with or without generating run-time code that checks the array boundaries at run time), the library interfaces are such that such compile time checks do not apply across API boundaries.

While interfaces like returntype functionname(size_t len, datatype data[len]) are supported since C99 –– data then has a variably modified type –– and compilers can detect simple overrun cases at compile time (and generate runtime bounds checking code) both for callers and callees using this kind of interface, they are extremely rare in practice: instead, returntype functionname(datatype *data, size_t len) is used, and this defeats the compile-time bounds checking, since the compiler cannot know that len is intended to be the length of the data array.  For the array slicing approach, only the latter (non-bounds-checkable) type of interface is possible.

One might argue that a language with C semantics, except replacing pointers with arrays so that pointers decay to arrays of size 1 with step zero, instead of arrays decaying to pointers to their first element, and with an optional standard library that uses the array slice approach, could be an obvious but relatively straightforward solution to the entire memory access and buffer "problem".

As to the actual generated machine code, one would still need a way to provide optimized implementations for specific step sizes for the same function, so some kind of conditional function overloading (which is not supported in C) would help.  Also, moving type, variable, and function (or scope) attributes into the syntax itself (perhaps similar to C++ [[attribute]] syntax) would be nice.

So, I don't really see that much wrong in C per se; it's mostly the C standard library that bugs me.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 06:49:09 pm
The prevalence of buffer overflow/underflow/overrun bugs in C is at least partly explained in that the fundamental unit of reference in the libraries is an address instead of an address range.

If we look at languages that natively support slicing, they conceptually use tuples (origin, step, length) or (origin, step, min-index, max-index) to describe array slices.

While current C compilers can check for array boundaries at compile time (with or without generating run-time code that checks the array boundaries at run time), the library interfaces are such that such compile time checks do not apply across API boundaries.

While interfaces like returntype functionname(size_t len, datatype data[len]) are supported since C99 –– data then has a variably modified type –– and compilers can detect simple overrun cases at compile time (and generate runtime bounds checking code) both for callers and callees using this kind of interface, they are extremely rare in practice: instead, returntype functionname(datatype *data, size_t len) is used, and this defeats the compile-time bounds checking, since the compiler cannot know that len is intended to be the length of the data array.  For the array slicing approach, only the latter (non-bounds-checkable) type of interface is possible.

One might argue that a language with C semantics, except replacing pointers with arrays so that pointers decay to arrays of size 1 with step zero, instead of arrays decaying to pointers to their first element, and with an optional standard library that uses the array slice approach, could be an obvious but relatively straightforward solution to the entire memory access and buffer "problem".

As to the actual generated machine code, one would still need a way to provide optimized implementations for specific step sizes for the same function, so some kind of conditional function overloading (which is not supported in C) would help.  Also, moving type, variable, and function (or scope) attributes into the syntax itself (perhaps similar to C++ [[attribute]] syntax) would be nice.

So, I don't really see that much wrong in C per se; it's mostly the C standard library that bugs me.

Well C is basically inextensible, new keywords pretty much cannot be added and so right there it's grammar becomes almost frozen. Of course most if not all modern languages suffer from this and - to a large degree - that is because they regard C as a reasonable starting point for their own grammar.

C places very little restriction on what one can do with/to a pointer, we can even change it programmatically to an illegal value, that's likely the Achilles heel of the language.

Instead thought should be given to why does someone want to change a pointer? what is they want to achieve by doing so? there are other ways to achieve things without needing the freedom to arbitrarily adjust or create, pointers.



Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 24, 2022, 07:24:18 pm
I don't really get the point of this keyword thing. Of course any language will limit the number of predefined keywords to avoid making it increasingly hard not to clash with users' identifiers.
OTOH, if you define very little to (at the extreme) no keyword in the language, you'll have to resort to cryptic base constructs using 'symbols' for your language syntax. This has been done before. This usually isn't pretty, but hey, beauty is in the eye of the beholder.

Not sure what you mean by "extensible". There have been hundreds of different ways of making a language "extensible", with various levels of success. Could you give examples of "extending", for instance, C, that would be actually useful? Extending the language on the syntax level is a slippery slope. It means basically that your language would be a constant moving target. Rarely a good thing. Now if by that you rather mean some macro system that is more elaborate than just text substitution, then sure. Why not. You'll find that in other languages already. Certainly the C preprocessor is very crude. But it's very flexible at the same time. So beware of trying to design a better macro system that would replace it: to get half of its flexibility, you're very likely to end up with an extremely complex system.

Regarding pointers, that's one of the key flexibility of C, but of course with power comes responsibility, as they say. Nobody forces you to use pointers in weird ways. You can even decide not to do any pointer arithmetic at all if you feel uncomfortable with that, and still write useful C.

Reading this thread, it all looks like you (and others) are pointing out roughly the same things that have been over and over again. I don't want to get you discouraged, but I'm failing to see what could be possibly new here.

While creating a new language is something that piques the interest of many if not most people writing software at some point in their life, most end up giving up and getting back to writing software. Getting around/doing with C quirks usually end up being more pragmatic and more productive than trying to create your own language, which in the end may not prove any better.

Or, you do as DiTBho describes. You write your own tools to fit your own needs, in which case you don't ask for others' opinions because they will not get you anywhere anyway - that would be like asking people what their favorite color is in order for you to pick the color of your next car. Just a thought.

I did create a "design a better C" thread quite a while ago as an experiment, and while there were a few interesting points, it went nowhere as expected. People more or less all have gripes about C, but when it comes to defining what exactly would make it better, it's suddenly a lot of silence, with the expected occasional "use Rust instead". ::)
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 08:26:06 pm
I don't really get the point of this keyword thing. Of course any language will limit the number of predefined keywords to avoid making it increasingly hard not to clash with users' identifiers.
OTOH, if you define very little to (at the extreme) no keyword in the language, you'll have to resort to cryptic base constructs using 'symbols' for your language syntax. This has been done before. This usually isn't pretty, but hey, beauty is in the eye of the beholder.

Not sure what you mean by "extensible". There have been hundreds of different ways of making a language "extensible", with various levels of success. Could you give examples of "extending", for instance, C, that would be actually useful? Extending the language on the syntax level is a slippery slope. It means basically that your language would be a constant moving target. Rarely a good thing. Now if by that you rather mean some macro system that is more elaborate than just text substitution, then sure. Why not. You'll find that in other languages already. Certainly the C preprocessor is very crude. But it's very flexible at the same time. So beware of trying to design a better macro system that would replace it: to get half of its flexibility, you're very likely to end up with an extremely complex system.

Regarding pointers, that's one of the key flexibility of C, but of course with power comes responsibility, as they say. Nobody forces you to use pointers in weird ways. You can even decide not to do any pointer arithmetic at all if you feel uncomfortable with that, and still write useful C.

Reading this thread, it all looks like you (and others) are pointing out roughly the same things that have been over and over again. I don't want to get you discouraged, but I'm failing to see what could be possibly new here.

While creating a new language is something that piques the interest of many if not most people writing software at some point in their life, most end up giving up and getting back to writing software. Getting around/doing with C quirks usually end up being more pragmatic and more productive than trying to create your own language, which in the end may not prove any better.

Or, you do as DiTBho describes. You write your own tools to fit your own needs, in which case you don't ask for others' opinions because they will not get you anywhere anyway - that would be like asking people what their favorite color is in order for you to pick the color of your next car. Just a thought.

I did create a "design a better C" thread quite a while ago as an experiment, and while there were a few interesting points, it went nowhere as expected. People more or less all have gripes about C, but when it comes to defining what exactly would make it better, it's suddenly a lot of silence, with the expected occasional "use Rust instead". ::)

Briefly, by extensible I mean trivial to freely add new keywords and attributes without breaking backward compatibility. The language grammar must be designed to meet that requirement else its either impossible or cumbersome at best. C's grammar is poor, it was the result of an exercise in minimalism and parser simplicity.

Examples? OK what about label variables an aspect of computed goto (this is pseudo code)

Code: [Select]

goto target[index];

target(1):
    // code

target(2):
   // code


That's a change to the goto syntax that likely won't be feasible for C. Or how about arrays of label variables? we could then alter the entries in that array and alter the behavior of "goto label[X];" this has applications in state machines.

Or iterators like we see in C#

Code: [Select]

iterator int get_randoms()
{
   for (int I=0; I < 100; I++)
        {
           yield random(I);
        }
}

main()
{

   foreach (R in get_random())
      {
      // do stuff
      }

}

In that example we have an expanded functionality for a function declaration and a new keyword "foreach". These are random examples and we can easily dream up more, things that would help a developer but in C its impossible to add these without being unable to compile earlier code.

As for pointers, we don't manipulate pointers for no reasons, there is always some ultimate objective. It is to either access data or update data. There are better ways to provide access just as flexibly without the crude unconstrained pointer concept, pointers are fine but there are superior ways to work with them, other ways that are less risky.

I have real experience with compiler design and implementation so my interest stems from that, I wrote a sophisticated compiler for the PL/I language in C, so I know those two languages well, that was a very interesting few years and I learned a lot about the good, the bad and the ugly with languages!

My position is that C has had a bad influence on languages, it has contaminated the grammars of a huge number of later languages. If we were designing a high level language for MCUs from scratch, there'd be no need, no value, in even considering C. It has given people "tunnel vision" they can often not see that there are other ways, other ideas.

Look at Ada, did you know that Nvidia recently decided to start sidelining C and C++ for all future firmware development in favor of Ada?

https://www.adacore.com/nvidia (https://www.adacore.com/nvidia)

Quote
AdaCore is partnering with NVIDIA™ to implement Ada and SPARK programming languages for select safety-critical firmware used in their next-generation embedded systems platforms. Customer using products such as NVIDIA® Jetson™ for embedded artificial intelligence (AI) computing, NVIDIA DRIVE™ AGX for autonomous vehicle (AV) development, and Isaac™ for Robotics are able to use the same technology at the application level. At the core of NVIDIA’s embedded systems is a system on a chip (SoC) that integrates an ARM architecture central processing unit (CPU), graphics processing unit (GPU), memory controller and IO paths onto one package. AdaCore’s GNAT Pro Ada and SPARK compilers can compile code targeting the ARM CPU embedded on NVIDIA SoCs, and increase verification efficiencies to achieve compliance with the functional safety standard ISO-26262. Supported compilation modes include bare metal as well as Linux or QNX OSes.









Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 24, 2022, 08:42:37 pm
but in C its impossible to add these without being unable to compile earlier code.
This is not true and C is expanded all the time. C specification clearly defines what identifiers (stuff that starts with underscore and a capital letter, and a number of other cases) are reserved for future use and those are used to expand the language.

The features you want (like iterators) would imply mandatory heap allocation and other behaviour that C avoids because it is not possible on all platforms (including small PICs).

It is good to want to have those high level language features in a low leve language, but it is impossible or impractical to actually implement them. So, when typing out the sample code you want to work, also type out the assembly you want that to translate into to see potential problems.

And even when things may conflict, C standard authors may still introduce new things after assessment of potential impact. For example C23 would introduce a new keyword "constexpr", which would make some old code break.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 24, 2022, 08:50:29 pm
Well that's fine, at least you've looked at it, its a rich subject and a huge source of healthy disagreement!

Speaking of disagreement, you say "I ignored C# since it offers nothing over Java (and introduces significant problems too)".

I'd be interested in hearing more on the problems you found.

Easy. Once you have a keyword "unsafe", all bets are off.

I'm not a great fan of the ahead of time optimisation. I prefer HotSpot's ability to optimize what the code+data is actually doing. C#/C/C++ can only guess what will happen at runtime, which means it has to be either pessimistic or trust that the programmer has given it correct hints.

Howeve, that's not a clear unambiguous advantage to Java.

BTW my principal trajectory was Algol, C, Smalltalk, Objective-C, (rejected C++!), Java. But there are many many other languages I've used, from mathematical modelling, hardware description languages, stats languages, and more.

I first came across C# before it was released,  when Hejlsberg came and have a talk at the research labs. Despite being genetically and financially predisposed to tinker with new toys, I don't think anybody bothered with C#.

Quote
I'm not aware any linguistic advantage Java has over C#, Java is not standardized either whereas C# is defined by an international, non proprietary standard.

Java (because it is based on the C grammar) find its very hard to add new keywords and grammar changes, for example Java does not have properties, so one has to code get_ and set_ or some other contrivance.

Java does not have generics, in C# generics are instantiated at runtime, Java has a limited compile-time form of generics (type erasure) with consequent serious limitations.

C# has partial classes greatly simplifying system that rely in custom generated source code.

C# has indexers.

C# has ref/out argument passing and supports optional arguments and positional arguments, not so Java.

C# has superb interop capabilities Java has very little to offer.

C# offers pointers for scenarios where code must interact with memory not managed by the CLR.

C# supports iterators, async iterators, async and await as language concepts.

C# has come a long way over the past twenty years, my only criticism of C# is that it might be getting a bit unwieldy and some of that is due to (IMHO) the fact that it too, presumed the C grammar rather than a fresh clean sheet of paper approach.

Also you say "syntax isn't very interesting" and I don't know what that means, do you mean you don't care at all about notational aspects of a language? We reason about code by looking at it, a good syntax improves one's ability to reason over a poor syntax.

This is why mathematics succeeds, it us superb when it comes to notation, we can look at some mathematical equation and reason about it, but express the same equation in assembler or C and we lose that ability to a great extent.

The king of languages from the standpoint of notation is APL, initially that was an experiment in a new mathematical notation for certain types of problems, later it became a programming language and remains to this day the pinnacle of expressivity.

Iverson - who invented APL - wrote a superb paper on this, it was a talk he gave after receiving the Turing Award for his contribution to computer science, syntax is everything to Iverson.

Notation as a tool of thought (https://www.jsoftware.com/papers/tot.htm).

In a similar way lambda calculus is better at expressing certain ideas than a Turing machine is.

International standardization is a red herring, and not a deciding factor. For anything complex each implementation will have its own quirks.

I hate language extensions for many reasons, especially because I (and toolsets) have to comprehend all the code in order.to understand what a+b means. The correct way to extend functionality is through libraries implemented in a simple language.

Getters and setters vs properties is mere syntactic sugar. Completely boring, and so trivial that tools do it for you.

Async and await have equivalents in Java lobraries, based on different primitives. No practical difference there.

I certainly don't think that Java is perfect; it isn't :) E.g its generics are useful in limited circumstances. And it too is suffering from becoming extended too much by second rate software weenies ;)

But the main point is that C# doesn't offer compelling advantages. Plus it doesn't run well on Linux systems (I'm not clever enough to be a season for Windows boxes!)
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 24, 2022, 09:17:23 pm
So before making sweeping pronouncements like "8 bit PIC is utterly unsuited to running any kind of modern high level compiled language" you need to ask yourself why Microchip are seemingly unaware of this.

I can assure you, Microchip is painfully well aware of this.

Would you say then that this could also apply to them "it is becoming more and more apparent that they don't know the first thing about either CPU instruction sets or programming languages, and they won't listen to those who do".
If you are going to insult the fine Scottish people of the 1970s like that, you are clearly a troll.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 09:47:23 pm
but in C its impossible to add these without being unable to compile earlier code.
This is not true and C is expanded all the time. C specification clearly defines what identifiers (stuff that starts with underscore and a capital letter, and a number of other cases) are reserved for future use and those are used to expand the language.

Well yes, C does have reserved words, I said that. Ideally a language would not need to reserve any words at all, Rust has unused reserved words too, pretty much any language derived from C must.

Just to be clear too the term "reserved words" usually means, simply "set aside, cannot be used for identifiers" and that's what I mean by the term. The Rust language defines it as word that are not yet recognized language keywords but might be in the future.

Consider a "yield" keyword, in C we are allowed to use a parenthesized expression after return:

Code: [Select]

   return (result);


If we wanted to add an alternative, a "yield" we'd want symmetry, we want to therefore allow:

Code: [Select]

   yield (item);


But we cannot, because if the user's code had a function name "yield" accepting a single argument, then the compiler is stuck.

The same problem faced the C# language team, they resolved the problem by making the yield syntax:

Code: [Select]

   yield return (item);


They call "yield" a contextual keyword. So we can declare a name "yield" and C# is fine, but they did have to contrive a syntax to support the yield as a keyword.

In a language with no reserved words at all, it is trivial, we can pick any new keyword we like and just use it, so in PL/I it would be a no brainer to allow:

Code: [Select]

   yield (item);


The underscore and capital letter scheme too, is an ugly contrivance, all they did there is choose a form that makes it unlikely to be used by a developer. But think about it, how can you anticipate in 1972 all of the potential, likely, future keywords your language will need? You cannot.

The features you want (like iterators) would imply mandatory heap allocation and other behaviour that C avoids because it is not possible on all platforms (including small PICs).

First I do not "want" iterators, I used it as an example when discussing grammars. Second what you say may or may not be true, but that's an implementation question, I was not discussing how that might be implemented only how the syntax could look. The implementation might be feasible or it might not, but that's nothing to do with the grammar, "parseability" of the language.

It is good to want to have those high level language features in a low leve language, but it is impossible or impractical to actually implement them. So, when typing out the sample code you want to work, also type out the assembly you want that to translate into to see potential problems.

And even when things may conflict, C standard authors may still introduce new things after assessment of potential impact. For example C23 would introduce a new keyword "constexpr", which would make some old code break.


Implementation is indeed important, if I had been suggesting specifically to add iterators then I'd agree with you 100%, but it was just an example of a keyword that one could add.

Making old code break is absolutely unacceptable to me, it is one of the primary goals of any new language for me. 100% backward compatibility is a core goal. Being able to compile ten year old code with a new version of a language that has eight more keywords than the earlier version and guaranteed a clean compile is - IMHO - non negotiable, not if one takes language design seriously.




Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 10:00:48 pm
PL/I (yes, he's on about that again) had a neat way to support language functions. These too could be added without breaking older source code. Inherent language functions "part of the language" were referred to as "builtin" functions. Consider:

Code: [Select]

a = substring("this can't be real",4);


The "substring" is "part of" the language. But if as user created a function named, say "frameptr" that took some argument, like this:

Code: [Select]

f = frameptr(4); // get pointer to the 4th window frame.


Then there's a problem. If the language team ever wanted to add a language function "frameptr" we're stuck, the new compiler will have a problem resolving the user's own function from the new language function!

This is resolved elegantly though. The approach is that all use of language functions must be first declared, like this:

Code: [Select]

dcl frameptr builtin; // dcl the function to walk up the stack frames.

// other code...

So to use any, even new, language functions one must declare them as "builtin" and since the language knows the signature, there's no need to write anything other than "builtin" the return type and argument type's don't need to be specified.

This is very powerful and goes hand in hand with the absence of reserved words to absolutely guarantee 100% backward compatibility with any new keyword or any new language function.

Builtin functions ordinarily includes stuff like string functions, mathematics functions and so on. So to use some of these in your code you'd write:

Code: [Select]

dcl cos          builtin;
dcl sin          builtin;
dcl substring    builtin;
dcl max          builtin;
dcl sqrt         builtin;






Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 24, 2022, 10:06:45 pm
OK what about label variables an aspect of computed goto
Already supported in C by GCC, Clang, and Intel CC at least, via labels as values (https://gcc.gnu.org/onlinedocs/gcc/Labels-as-Values.html#Labels-as-Values).  For example,
Code: [Select]
int is_even(int x)
{
    static const void *const target[2] = { &&even, &&odd };

    goto *target[x & 1];

odd:
    return 0;

even:
    return 1;
}

Or iterators like we see in C#
Those can be implemented via C coroutine (https://en.wikipedia.org/wiki/Coroutine#C) support.  Granted, you need either POSIX or SuS support, or an assembly (machine code) implementation, to be able to obtain and swap stack and register file context, but many library implementations for C do exist.

Essentially, "yield()" and "next()" form a coroutine pair, where calling one switches execution to the other.

In a low level language, the implicit context swap would be a Big Problem.  It is like exceptions in C++, which tend to be too resource intensive to be worth implementing in microcontroller environments.  In a microcontroller, available stack is often counted in bytes, so having two or more stacks at the same time is a no-no.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 10:10:56 pm
OK what about label variables an aspect of computed goto
Already supported in C by GCC, Clang, and Intel CC at least, via labels as values (https://gcc.gnu.org/onlinedocs/gcc/Labels-as-Values.html#Labels-as-Values).  For example,
Code: [Select]
int is_even(int x)
{
    static const void *const target[2] = { &&even, &&odd };

    goto *target[x & 1];

odd:
    return 0;

even:
    return 1;
}

Or iterators like we see in C#
Those can be implemented via C coroutine (https://en.wikipedia.org/wiki/Coroutine#C) support.  Granted, you need either POSIX or SuS support, or an assembly (machine code) implementation, to be able to obtain and swap stack and register file context, but they do exist.  Essentially, "yield()" and "next()" form a coroutine pair, where calling one switches execution to the other.

In a low level language, the implicit context swap would be a Big Problem.  It is like exceptions in C++, which tend to be too resource intensive to be worth implementing in microcontroller environments.  In a microcontroller, available stack is often counted in bytes, so having two or more stacks at the same time is a no-no.

Yes, there are various vendor specific changes to C and this is one of them, but it's rather a bit of a bolt-on job. Here's how it could look:

Code: [Select]

   goto target(x+1);

target(0):
   return 0;

target(1):
   return 1;


Yes, yield and iterators can be viewed as a coroutine concept, I did call out coroutine support too in my earlier list of features. Also how something might be implemented is a valid conversation but I'm mainly talking about grammars just now.

If something cannot reasonably be accommodated then one would not implement the feature, but until that's been determined one can consider the feature as a possibility.
Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 24, 2022, 10:25:35 pm
You are focusing way too much on "nice looking" syntax. This is not the most important part. If your code is full of those computed gotos, then you are likely doing something wrong anyway.

And I'm lost again what is being discussed. You throw some random examples of code that you don't want to work.

Yes, it is possible to make a language exactly as you like, but that happens by opening a text editor and typing the code, not forum posts.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 10:38:02 pm
You are focusing way too much on "nice looking" syntax. This is not the most important part. If your code is full of those computed gotos, then you are likely doing something wrong anyway.

And I'm lost again what is being discussed. You throw some random examples of code that you don't want to work.

Yes, it is possible to make a language exactly as you like, but that happens by opening a text editor and typing the code, not forum posts.

You could be contributing to the conversation (instead of trolling) as several other people are, I'm sure you have much to offer but it seems you simply resent me discussing the subject. Several posts have contained genuinely useful responses, things that give me pause for thought.

I will discuss what I want to discuss with whomever wants to discuss it with me, your approval or disapproval of the subject matter is unimportant.

Oh, and regarding the naïve advice "opening a text editor and typing the code, not forum posts" is a common failing; starting to write software to solve a problem before that problem is defined, is the cause of many of today's ills in this business.



Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 24, 2022, 10:39:16 pm
So before making sweeping pronouncements like "8 bit PIC is utterly unsuited to running any kind of modern high level compiled language" you need to ask yourself why Microchip are seemingly unaware of this.

I can assure you, Microchip is painfully well aware of this.

Would you say then that this could also apply to them "it is becoming more and more apparent that they don't know the first thing about either CPU instruction sets or programming languages, and they won't listen to those who do".
If you are going to insult the fine Scottish people of the 1970s like that, you are clearly a troll.

Some variety of small furry animal for sure.

And I think you mean "Pict".
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 24, 2022, 10:42:28 pm
So before making sweeping pronouncements like "8 bit PIC is utterly unsuited to running any kind of modern high level compiled language" you need to ask yourself why Microchip are seemingly unaware of this.

I can assure you, Microchip is painfully well aware of this.

Would you say then that this could also apply to them "it is becoming more and more apparent that they don't know the first thing about either CPU instruction sets or programming languages, and they won't listen to those who do".
If you are going to insult the fine Scottish people of the 1970s like that, you are clearly a troll.

Some variety of small furry animal for sure.

And I think you mean "Pict".
The PIC was developed in Glenrothes. I think the Picts were farther north than that.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 10:44:53 pm
So before making sweeping pronouncements like "8 bit PIC is utterly unsuited to running any kind of modern high level compiled language" you need to ask yourself why Microchip are seemingly unaware of this.

I can assure you, Microchip is painfully well aware of this.

Would you say then that this could also apply to them "it is becoming more and more apparent that they don't know the first thing about either CPU instruction sets or programming languages, and they won't listen to those who do".
If you are going to insult the fine Scottish people of the 1970s like that, you are clearly a troll.

I am Scots, even we had trouble with the picts - or is it pickets...I forget...

Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 24, 2022, 10:46:33 pm
(instead of trolling)
I'm not trolling, I've just seen this exact thread dozens of times. Nothing practical will come out of it.

If the goal is to get generally educated on why certain compilers do something one way or the other, I'm perfectly fine with that. But you start your claims with stuff like "C can't do this" or "PIC authors don't know", which are all incorrect.

And that discussion of Scots is sure on-topic here.

Sure, having computed gotos with your syntax would be nice. What's next?
Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 24, 2022, 10:49:26 pm
Here's how it could look:
Me no like.  How do you do range checking on x?  And the syntax looks too much like function calls to me.  The colon at the end is so easy to confuse with a semicolon.  Obfuscated C material, methinks...

Anyway, I have used the labels-as-values for state machines.  However, I do prefer a function pointer variant, with each state or transition (depending on how you define your state machine and events) being a separate function call, instead of a single huge switch() or goto/label mess.

For things like command interfaces –– basically name-to-function mappings ––, I like to use ELF section support.  (All architectures I use support ELF object files.)  Basically, a preprocessor macro is used to generate a static const structure in a dedicated section (https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#index-section-variable-attribute).  The linker will collect these at link time from all object files, and concatenate them into a single array, and the start and end addresses are exported by symbols visible to the C code.
Note: This is not a C feature, but a feature provided by C compilers and linkers when they use ELF format object files.

This means that if I write a G-code interpreter, I can use a dedicated section for supported G-codes, and another for M-codes.  In the source code, anywhere a new one is defined, it gets declared using something like DECLARE_G(code, function, "description") or DECLARE_M(code, function, "description"), and the interpreter will look up each one using the linker-provided section symbols.

This works amazingly well.  I can put each feature into a separate source file, and let the builder select the desired set at compile time.  The only thing that annoys me is that the linker does not yet know how to sort those entries at link time.  I often have a key, here code (a positive integer), and if the array entries were sorted according to the key, I could use a binary search to find the entry in O(log N) time.  If I want to do that now, I need to do a separate post-link object file modification step.  And if I have to do that, I can spend a little more time doing that, and make the array into a hash table with the hash function provided by the post-link step, to find a near-perfect hash function for the data, for O(L) time lookups, where L is the maximum collision chain length.

You are focusing way too much on "nice looking" syntax. This is not the most important part. If your code is full of those computed gotos, then you are likely doing something wrong anyway.

And I'm lost again what is being discussed. You throw some random examples of code that you don't want to work.

Yes, it is possible to make a language exactly as you like, but that happens by opening a text editor and typing the code, not forum posts.
You could be contributing to the conversation (instead of trolling) as several other people are
I do not always agree with ataradov, but here, he has a particularly apt point.

The way the code looks is not important.  Readability is important, but "target(0):" is less readable than "target_0" because it is so easy to confuse with "target(0);", so you are demonstrably focusing on the wrong thing, at least in this example.

What is important, is how easy or hard it is to describe an expression or concept in the given language with exactly the desired side effects and no others.
This means that the most important things to achieve with a new low-level language with the intent of replacing C, is to be able to express everything that can be expressed in C already, and more, or at least better, to some definition of "better".  "Visually more appealing" is not "better", though.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 24, 2022, 10:52:59 pm
(instead of trolling)
I'm not trolling, I've just seen this exact thread dozens of times. Nothing practical will come out of it.

If the goal is to get generally educated on why certain compilers do something one way or the other, I'm perfectly fine with that. But you start your claims with stuff like "C can't do this" or "PIC authors don't know", which are all incorrect.

You're trying to stifle conversation, stifle dissent, you disapprove of someone who sees things differently to you, that's pretty much it isn't it Alex?. Well not here, that's the stuff of Orwell, 1984, the film Brazil, intolerant, repressive communist regimes, but not here.

So I don't care what you're "perfectly OK with" that's not of interest to me.

And that discussion of Scots is sure on-topic here.

I didn't bring it up buddy.

Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 24, 2022, 10:54:32 pm
Look at Ada, did you know that Nvidia recently decided to start sidelining C and C++ for all future firmware development in favor of Ada?

https://www.adacore.com/nvidia (https://www.adacore.com/nvidia)

That seems reasonable. Much more so than switching to uh, Rust, for instance. ;D
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 24, 2022, 10:58:27 pm
Look at Ada, did you know that Nvidia recently decided to start sidelining C and C++ for all future firmware development in favor of Ada?

https://www.adacore.com/nvidia (https://www.adacore.com/nvidia)
That seems reasonable. Much more so than switching to uh, Rust, for instance. ;D
Maybe they are just trying to sidestep ageism, or create opportunities for ex-defence people, and hire some people old enough to have actually used Ada.  :)
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 24, 2022, 11:14:51 pm
Did no one get my troll ... Pict reference? :-( :-(
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 24, 2022, 11:39:14 pm
Did no one get my troll ... Pict reference? :-( :-(
Have you been grooving with one?
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 25, 2022, 01:12:01 am
Look at Ada, did you know that Nvidia recently decided to start sidelining C and C++ for all future firmware development in favor of Ada?

https://www.adacore.com/nvidia (https://www.adacore.com/nvidia)

That seems reasonable. Much more so than switching to uh, Rust, for instance. ;D

Hot off the press: "The U.S. NSA finally came out this week to strongly endorse `memory-safe' languages for most software programming, specifically mentioning C#, Go, Java, Ruby, Rust, and Swift as examples. Apparently orphaned DoD language *Ada* was conspicuously left out of NSA's list, even though versions of Ada that target JVM can utilize Java JVM's GC."
FFI see http://catless.ncl.ac.uk/Risks/33/53#subj3 (http://catless.ncl.ac.uk/Risks/33/53#subj3)

If a language doesn't reduce the chances of something appearing in that comp.risks, then I'm unlikely to want to swap to it.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 25, 2022, 01:15:24 am
You are focusing way too much on "nice looking" syntax. This is not the most important part. If your code is full of those computed gotos, then you are likely doing something wrong anyway.

And I'm lost again what is being discussed. You throw some random examples of code that you don't want to work.

Yes, it is possible to make a language exactly as you like, but that happens by opening a text editor and typing the code, not forum posts.

You could be contributing to the conversation (instead of trolling) as several other people are, I'm sure you have much to offer but it seems you simply resent me discussing the subject. Several posts have contained genuinely useful responses, things that give me pause for thought.

I too have stated that I think sugared syntax is relatively unimportant; clean and orthogonal and understandable semantics are much more important.

I disagree that ataradov is trolling. Blunt != trolling in any sense of the word.
Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 25, 2022, 01:25:09 am
Look at Ada, did you know that Nvidia recently decided to start sidelining C and C++ for all future firmware development in favor of Ada?

https://www.adacore.com/nvidia (https://www.adacore.com/nvidia)

That seems reasonable. Much more so than switching to uh, Rust, for instance. ;D

Hot off the press: "The U.S. NSA finally came out this week to strongly endorse `memory-safe' languages for most software programming, specifically mentioning C#, Go, Java, Ruby, Rust, and Swift as examples. Apparently orphaned DoD language *Ada* was conspicuously left out of NSA's list, even though versions of Ada that target JVM can utilize Java JVM's GC."
FFI see http://catless.ncl.ac.uk/Risks/33/53#subj3 (http://catless.ncl.ac.uk/Risks/33/53#subj3)

If a language doesn't reduce the chances of something appearing in that comp.risks, then I'm unlikely to want to swap to it.

The list above is hilarious. Uh and advice from the NSA? Not interested. :-DD
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 25, 2022, 03:36:35 am
Did no one get my troll ... Pict reference? :-( :-(
Have you been grooving with one?

In a cave.

Have you seen the Simpsons lost in the forest at night mashup, with Willie as the Pict?
Title: Re: A new, hardware "oriented" programming language
Post by: Siwastaja on November 25, 2022, 08:56:41 am
Look at Ada, did you know that Nvidia recently decided to start sidelining C and C++ for all future firmware development in favor of Ada?

To be fair, take these "we decided to start doing X" things with grain of salt. It's similar to Toyota deciding on transition into hydrogen cars every year for last 30 years.

Chances are pretty high it's just a PR program and maybe some trainee replaces some (unused or unimportant) module, while actual work commences just like before.

Professionals who enable all this crazy hightech are just fine working with C, after all, and don't like managers messing up their good productivity for social scores.
Title: Re: A new, hardware "oriented" programming language
Post by: alexanderbrevig on November 25, 2022, 12:25:51 pm
[Professionals who enable all this crazy hightech are just fine working with C, after all, and don't like managers messing up their good productivity for social scores.

Productively producing bugs that could've been avoided. No business likes that. I rarely meet developers who like maintaining and bugfixing C/C++ when the fun challenge of getting it stable is first completed.

With Rust you will typically find libraries that have no activity for a year or two. That's a huge warning sign with most other languages. With Rust, it simply means it's done.
Think about that! DONE!

We have an in house component written in Rust and the only times we've had to change the container is when there's CVEs in the underlying container runtime. Nothing has been reported in my dependencies, and no bugs have been reported by users. Two years in production. Let's just say that every other component has a long list of issues against them.
It's a brave, new and delightful world!
Title: Re: A new, hardware "oriented" programming language
Post by: DiTBho on November 25, 2022, 12:41:52 pm
Well, do178b/misra95 C code has the same "it's done" characteristic.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 25, 2022, 01:09:21 pm
Well, do178b/misra95 C code has the same "it's done" characteristic.

Both that and Rust are C with the naughty bits forbidden  >:D Ditto SPARK and Ada :)

No, I'm not going to attempt to fully justify those statements, but there is some truth in them.

Removing naughty bits is far more important that fiddling with syntax. Unless, of course, it is the syntax that is the source of "inappropriate" behaviour.
Title: Re: A new, hardware "oriented" programming language
Post by: DiTBho on November 25, 2022, 01:57:54 pm
Removing naughty bits is far more important that fiddling with syntax. Unless, of course, it is the syntax that is the source of "inappropriate" behaviour.

Misra is about synthax and grammar: necessary (for ICE inspections and analysis) but not sufficient.
Do178b is about passing (by ICE validation) the appropriate behavior(1): suffient.
Combined && -> QA passed

(1) normal, abnormal, defensive code
Title: Re: A new, hardware "oriented" programming language
Post by: Siwastaja on November 25, 2022, 02:09:19 pm
Productively producing bugs that could've been avoided.

Language purists are interesting folks, because they promise you the Moon from the sky, and completely bug-free systems. Compare that to non-programming fields! How about an aircraft engineer who claims they never make any mistakes? Anyone heard about such ridiculous claim?

For some strange reason, the track record for these "perfect" languages is near-zero. It's always "C causes bugs, <X> is perfectly bug-free, oh why are all the professionals so stupid they don't use <X> but stick to C, boo hoo hoo".

Talk is cheap. C is indeed quite crappy but most of the dangerous aspects are very well known to the professionals and avoided. They know their tools, and despite all the shortcomings of C, its strongest points are standardization, the fact it's not a moving target, being very well known, something to build expertise on.

Surgeons also use knives despite the fact you need to use them carefully. There are no wannabe-surgeons who complain all the time and have little to show, though. That's the difference when compared to computer programming.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 02:41:15 pm
Productively producing bugs that could've been avoided.

Language purists are interesting folks, because they promise you the Moon from the sky, and completely bug-free systems. Compare that to non-programming fields! How about an aircraft engineer who claims they never make any mistakes? Anyone heard about such ridiculous claim?

For some strange reason, the track record for these "perfect" languages is near-zero. It's always "C causes bugs, <X> is perfectly bug-free, oh why are all the professionals so stupid they don't use <X> but stick to C, boo hoo hoo".

Talk is cheap. C is indeed quite crappy but most of the dangerous aspects are very well known to the professionals and avoided. They know their tools, and despite all the shortcomings of C, its strongest points are standardization, the fact it's not a moving target, being very well known, something to build expertise on.

Surgeons also use knives despite the fact you need to use them carefully. There are no wannabe-surgeons who complain all the time and have little to show, though. That's the difference when compared to computer programming.

You say "talk is cheap" yet that entire post is nothing but anecdotes, how about facts, metrics, objective data?

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 02:47:11 pm
This is one of two forums where this subject is being actively discussed for MCU languages, I think I have enough at this stage to begin to define the formal grammar for such a language.

The grammar doesn't cover semantics and other specifics but will be enough to perhaps get a working recursive descent parser up and running, if I do venture that far I'll be writing this in C# for .Net 7 since that is highly portable now, a parsing tool will thus run fine on either Windows or Linux.

Perhaps with a working initial grammar some of this discussion will become less abstract.

I already have a very reasonable lexical analysis library I put together last year, this is a small FSM that's driven by a CSV (https://github.com/Steadsoft/novus/blob/main/Steadsoft.Novus.Scanner/Classes/Tokenizer.cs#L144) token definition file, for those curious you can see this here.

Scanner library. (https://github.com/Steadsoft/novus/tree/main/Steadsoft.Novus.Scanner)

CSV Token Definitions Example (https://github.com/Steadsoft/novus/blob/main/Sandbox/Novus/novus.csv).





Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 02:58:17 pm
Look at Ada, did you know that Nvidia recently decided to start sidelining C and C++ for all future firmware development in favor of Ada?

https://www.adacore.com/nvidia (https://www.adacore.com/nvidia)

That seems reasonable. Much more so than switching to uh, Rust, for instance. ;D

Hot off the press: "The U.S. NSA finally came out this week to strongly endorse `memory-safe' languages for most software programming, specifically mentioning C#, Go, Java, Ruby, Rust, and Swift as examples. Apparently orphaned DoD language *Ada* was conspicuously left out of NSA's list, even though versions of Ada that target JVM can utilize Java JVM's GC."
FFI see http://catless.ncl.ac.uk/Risks/33/53#subj3 (http://catless.ncl.ac.uk/Risks/33/53#subj3)

If a language doesn't reduce the chances of something appearing in that comp.risks, then I'm unlikely to want to swap to it.

Well they are all memory safe, all of them only expose raw pointers as an option, only needed when interoperating with systems that rely on pointers like C or C++ libraries.

When not using raw pointers these language ecosystems can be 100% sure that invalid pointers/addresses can never ever arise at runtime.

Interesting mention of array bound checking, there are two forms of checking one is pretty fast the other is slow.

The fast one ensures memory safety whereas the slow one ensures memory safety and calling code validity.



Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 03:09:40 pm
Well, do178b/misra95 C code has the same "it's done" characteristic.

Both that and Rust are C with the naughty bits forbidden  >:D Ditto SPARK and Ada :)

No, I'm not going to attempt to fully justify those statements, but there is some truth in them.

Removing naughty bits is far more important that fiddling with syntax. Unless, of course, it is the syntax that is the source of "inappropriate" behaviour.

Bear in mind my own views here are primarily about grammar not syntax.

The core of the grammar should - IMHO - be along the lines of:

<source> ::= <assignment_stmt> ";" | <keyword_stmt>

<assignment_stmt> ::= <reference> = <expression>

<reference> ::= <qualified_name> | <qualified_name> [ "(" <commalist> ")"] etc..

<keyword_stmt> ::= <declaration> ; | <defintion> <def_body>

<declaration> ::= "declare" <identifier> [<array_def>] <reqd_dcl_attribs> [<optional_dcl_attribs>] etc

<definition> ::= <type_def> | <entry_def>

<entry_def> ::= <proc_def> | <func_def>

<proc_def> ::= "procedure" <identifier> [(<commalist>)] <reqd_proc_attribs> [<optional_proc_attribs>] "{" <proc_body> "}"

etc etc etc...
Title: Re: A new, hardware "oriented" programming language
Post by: DiTBho on November 25, 2022, 03:35:56 pm
You won't fix anything this way.
Necessary but not sufficient.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 25, 2022, 03:40:34 pm
Language purists are interesting folks, because they promise you the Moon from the sky, and completely bug-free systems. Compare that to non-programming fields! How about an aircraft engineer who claims they never make any mistakes? Anyone heard about such ridiculous claim?

Just so.

Quote

and despite all the shortcomings of C, its strongest points are standardization, the fact it's not a moving target, being very well known, something to build expertise on.

Except for all the parts that fall outside that category.

Isn't it obvious that all the implementation dependent behaviour isn't standardised?!

And then there's all the undefined behaviour, which trips up many many people.


Quote
Surgeons also use knives despite the fact you need to use them carefully. There are no wannabe-surgeons who complain all the time and have little to show, though. That's the difference when compared to computer programming.

Surgeons have to be rigorously trained and pass exams, and are regularly monitored by formal regulatory bodies. Anybody can call themselves a programmer and practice [sic] it without being monitored.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 25, 2022, 03:45:38 pm
You won't fix anything this way.
Necessary but not sufficient.

Precisely!

It seems to be axiomatic that digital hardware engineers yearn to create their own processor, and software engineers yearn to create their own language. Without a lot of industry knowledge plus clarity and insight, both are doomed to being (at best) an interesting academic exercise.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 04:34:17 pm
You won't fix anything this way.
Necessary but not sufficient.

Precisely!

It seems to be axiomatic that digital hardware engineers yearn to create their own processor, and software engineers yearn to create their own language. Without a lot of industry knowledge plus clarity and insight, both are doomed to being (at best) an interesting academic exercise.

Well whether some exercise is "doomed", depends entirely upon what one's objectives are. If the objectives are met then by definition one has success.

If I create a language, and it meets my overall objectives then I don't see how that can be regarded as an example of "doomed".

It is impossible to predict influences as well, one has no idea how an idea or a concept might propagate over time.

Perhaps you think I am attempting to create a commercial product, or establish a new language that achieves huge popularity, but that isn't the case at all.

Title: Re: A new, hardware "oriented" programming language
Post by: Siwastaja on November 25, 2022, 05:08:35 pm
You say "talk is cheap" yet that entire post is nothing but anecdotes, how about facts, metrics, objective data?

FWIW, to me the one single objective datapoint is I professionally develop firmwares for embedded devices in C and have absolutely no problem with it whatsoever and do not complain about it lacking anything critical, or generating critical bugs for me. I haven't stepped in any C footgun in years, just make classical bugs (wrong assumptions, logical mistakes) I would do in any language, and not too many. I also don't mind supporting my code, it's not fire-and-forget.

I know a few others in my field who also do not struggle. But I'm not going to give names, just like you are not giving the names of your "very experts" who struggle with PICs.
Title: Re: A new, hardware "oriented" programming language
Post by: Siwastaja on November 25, 2022, 05:18:10 pm
Isn't it obvious that all the implementation dependent behaviour isn't standardised?!

What weird mental acrobatics. It being in the standard means it is standardized! Standard basically says: we did not define the effect of this operation, so don't do that unless you really know what you are doing.

Would be very nice if implementation-defined behavior was not needed in the standard, but it's still much better to have standardized implementation-defined behavior than the standard being completely silent about some corner cases. This makes C standard good, compared to poorly standardized languages where you can't know what happens by reading the standard.

Quote
And then there's all the undefined behaviour

Exactly the same for UB! This is the strong point of C: UB being standardized, so just read and understand the standard, don't do UB, and you are good to go.

Having such standard is not obvious at all.

Quote
Surgeons have to be rigorously trained and pass exams, and are regularly monitored by formal regulatory bodies. Anybody can call themselves a programmer and practice [sic] it without being monitored.

But anyone can also buy surgical knives freely and do whatever with them. And doing something with C where human lives are at a stake, say design an autopilot for a passenger aircraft, also requires at least some form of monitoring by formal regulatory bodies. Anyone is free to use surgical knife to peel an apple, even if that poses a risk of cutting oneself. My surgeon analogue is thus valid.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 06:08:37 pm
You say "talk is cheap" yet that entire post is nothing but anecdotes, how about facts, metrics, objective data?

FWIW, to me the one single objective datapoint is I professionally develop firmwares for embedded devices in C and have absolutely no problem with it whatsoever and do not complain about it lacking anything critical, or generating critical bugs for me. I haven't stepped in any C footgun in years, just make classical bugs (wrong assumptions, logical mistakes) I would do in any language, and not too many. I also don't mind supporting my code, it's not fire-and-forget.

I know a few others in my field who also do not struggle. But I'm not going to give names, just like you are not giving the names of your "very experts" who struggle with PICs.

Right, so no facts to backup the original anecdotal remarks and now the rather obvious claim that there are many engineers who are quite content with C. None of that has any bearing on what's being discussed, obviously it is not those who are content, that inspire or generate progress.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 25, 2022, 06:18:02 pm
You say "talk is cheap" yet that entire post is nothing but anecdotes, how about facts, metrics, objective data?

FWIW, to me the one single objective datapoint is I professionally develop firmwares for embedded devices in C and have absolutely no problem with it whatsoever and do not complain about it lacking anything critical, or generating critical bugs for me. I haven't stepped in any C footgun in years, just make classical bugs (wrong assumptions, logical mistakes) I would do in any language, and not too many. I also don't mind supporting my code, it's not fire-and-forget.

I know a few others in my field who also do not struggle. But I'm not going to give names, just like you are not giving the names of your "very experts" who struggle with PICs.

Right, so no facts to backup the original anecdotal remarks and now the rather obvious claim that there are many engineers who are quite content with C. None of that has any bearing on what's being discussed, obviously it is not those who are content, that inspire or generate progress.

There are many engineers who are content with C.

There were many engineers who were content with COBOL. And steam engines. And bronze swords.

Nonetheless, incremental changes are legion (and ignorable), whereas significant advances are rare and to be enthusiastically accepted. It takes wide experience and good taste to distinguish one from the other.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 06:18:13 pm
Isn't it obvious that all the implementation dependent behaviour isn't standardised?!

What weird mental acrobatics. It being in the standard means it is standardized! Standard basically says: we did not define the effect of this operation, so don't do that unless you really know what you are doing.

Would be very nice if implementation-defined behavior was not needed in the standard, but it's still much better to have standardized implementation-defined behavior than the standard being completely silent about some corner cases. This makes C standard good, compared to poorly standardized languages where you can't know what happens by reading the standard.

Quote
And then there's all the undefined behaviour

Exactly the same for UB! This is the strong point of C: UB being standardized, so just read and understand the standard, don't do UB, and you are good to go.

Having such standard is not obvious at all.

Quote
Surgeons have to be rigorously trained and pass exams, and are regularly monitored by formal regulatory bodies. Anybody can call themselves a programmer and practice [sic] it without being monitored.

But anyone can also buy surgical knives freely and do whatever with them. And doing something with C where human lives are at a stake, say design an autopilot for a passenger aircraft, also requires at least some form of monitoring by formal regulatory bodies. Anyone is free to use surgical knife to peel an apple, even if that poses a risk of cutting oneself. My surgeon analogue is thus valid.

Well regarding the latter, avionics software development processes must be certified by the FAA as DO-178C compliant. For this reason most avionics software is written in Ada, not C, not C++.

Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 25, 2022, 06:44:48 pm
You say "talk is cheap" yet that entire post is nothing but anecdotes, how about facts, metrics, objective data?

FWIW, to me the one single objective datapoint is I professionally develop firmwares for embedded devices in C and have absolutely no problem with it whatsoever and do not complain about it lacking anything critical, or generating critical bugs for me. I haven't stepped in any C footgun in years, just make classical bugs (wrong assumptions, logical mistakes) I would do in any language, and not too many. I also don't mind supporting my code, it's not fire-and-forget.

I know a few others in my field who also do not struggle. But I'm not going to give names, just like you are not giving the names of your "very experts" who struggle with PICs.

Yeah, ditto.
Title: Re: A new, hardware "oriented" programming language
Post by: pcprogrammer on November 25, 2022, 07:24:27 pm
Seems like the python thread all over again.

A "better" programming language will not help you become a better programmer. It is not about the language, it is the mindset that makes a good programmer. The ability to understand why something does not work as intended and then fix it, is what counts.

For me plain C without libraries and HAL work very well on bare metal embedded projects. Yes you have to do everything yourself, but over time you build up your own set of code that can be used over and over again. The benefit of this is that you know what everything does.

I have seen things go sideways with for example C#, where a lot is done for you, and it can take a lot of effort to fix it when it turns out to be a bug in the libraries that come with it. Also had to work around bugs in javascript that can take a lot of time to figure out why things go wrong. Even with C++ and then perform bound checks, I have seen memory leaks within the libraries.

But like others wrote, if you like doing something like this go ahead, but I expect that not a lot of the expert embedded developers want to become testers of your new language.
Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 25, 2022, 07:44:26 pm
Would be very nice if implementation-defined behavior was not needed in the standard, but it's still much better to have standardized implementation-defined behavior than the standard being completely silent about some corner cases.
In my opinion, it is the implementation-defined behaviour that lets C evolve users first, as opposed to standards folk dictating the evolution to users and compiler developers.  It is closely intertwined with the C extensions, but I see the IB as giving leeway to change, and extensions as new growth.

Long-winded explanation of the basis of that opinion:

As I mentioned before, I use extensions to the C and C++ language, especially when they are supported by more than one compiler on the target hardware.
I also often rely on POSIX C (which adds to the standard library, but also requires specific behaviour in some places where the C standard itself is more relaxed).

In certain cases, I do rely on implementation-defined behaviour, when the compiler I use explicitly defines what its behaviour is.  It is okay, as long as the behaviour expectation is well described in the code, documentation, and build instructions.

In the case of GCC, for character set stuff, first check the preprocessor implementation-defined behaviour (and limits) (https://gcc.gnu.org/onlinedocs/cpp/Implementation-Details.html); the C compiler implementation-defined behaviour is defined here (https://gcc.gnu.org/onlinedocs/gcc/C-Implementation.html).

GCC implementation-defined behaviour I routinely rely on (but knowing it is implementation-defined):As to the extensions, I use statement expressions (https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html) in function-like macros to evaluate macro arguments only once (especially with the typeof (https://gcc.gnu.org/onlinedocs/gcc/Typeof.html) keyword extension); __float128 (https://gcc.gnu.org/onlinedocs/gcc/Floating-Types.html) (usually via libquadmath (https://gcc.gnu.org/onlinedocs/libquadmath/), though); array qualifiers work like pointer qualifiers (https://gcc.gnu.org/onlinedocs/gcc/Pointers-to-Arrays.html) so that type var[n]] can be used where a const type var[n] is expected; void * can consume any type of variadic pointer (https://gcc.gnu.org/onlinedocs/gcc/Variadic-Pointer-Args.html); initialization of objects with static storage duration by compound literals (https://gcc.gnu.org/onlinedocs/gcc/Compound-Literals.html) with constant elements; function (https://gcc.gnu.org/onlinedocs/gcc/Function-Attributes.html), variable (https://gcc.gnu.org/onlinedocs/gcc/Variable-Attributes.html), type (https://gcc.gnu.org/onlinedocs/gcc/Type-Attributes.html) attributes; fallthrough (https://gcc.gnu.org/onlinedocs/gcc/Statement-Attributes.html) hint in case statements; deprecated and unavailable (https://gcc.gnu.org/onlinedocs/gcc/Enumerator-Attributes.html) attributes in enums; '\e' as an alias for '\033' (https://gcc.gnu.org/onlinedocs/gcc/Character-Escapes.html); __alignof__ (https://gcc.gnu.org/onlinedocs/gcc/Alignment.html) in C90/C99 like C11 _Alignof; extended asm (https://gcc.gnu.org/onlinedocs/gcc/Using-Assembly-Language-with-C.html) (especially in static inline helper functions defined in header files); vector extensions (https://gcc.gnu.org/onlinedocs/gcc/Vector-Extensions.html); __atomic built-ins (https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html) with C++11-compatible memory model; other built-ins (https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html) like __builtin_constant_p(), __builtin_types_compatible(), __builtin_expect(), __builtin_assume_aligned(), __builtin_prefetch(); and __thread (https://gcc.gnu.org/onlinedocs/gcc/Thread-Local.html) for thread-local storage class keyword.

If we look at how the C standard evolved up to C99, implementation-defined behaviour let compiler developers and their users add new useful features.  For example, the __thread TLS keyword cannot be implemented without compiler support, but it makes a major difference to multithreaded programming in POSIX C.  C11 was an unfortunate sidestep, and C17 added nothing new, but C23 (https://en.wikipedia.org/wiki/C2x) seems to be back on sane track in my opinion.

Undefined behaviour is the "here be dragons" part of C: unexplored country.  Most of them are cases where the undefinedness stems from any arbitrary choice on the behaviour placing undue implementation limits on the compilers.  For example, when multiple pre- or post-increment or decrement operators are applied to the same variable in a single expression –– say, ((++x*y++)-(++x*++y)).  You could dictate a specific behaviour, but that would severely limit what kind of mechanisms C compilers use to evaluate such statements.  In particular, currently most use an Abstract Syntax Tree model, and apply expression optimization to the tree, before conversion to machine code; and it is likely that any left to right ordering is lost in the optimization phase.  If you dictate an order based on left-to-right positioning, it would require ASTs to add an ordering property, and any requirements related to that would likely conflict with the optimization passes.  No, better leave them undefined behaviour, unless someone comes up with a convincing use case that does not suffer from such.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 09:50:52 pm
There is no surprise that there are engineers quite content with C, or even committed to C. That is something I fully expect. But their presence does not somehow invalidate attempts to explore alternatives; that C has a huge following does not alter the fact that there are weaknesses in the language, that some weaknesses can be avoided by an improved language, and improved grammar.

Java is the new COBOL yet it too has a dedicated following, committed devotees.

This thread is simply aimed at those people who are interested enough in languages and pushing the envelope, to speculate on improvements and suggest ideas that might be relevant to a new MCU oriented language.

So I get it, I get that some consider this pointless, but some of us do not, so can we stop complaining about C being critiqued and press on with positive contributions?


Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 09:58:07 pm
Seems like the python thread all over again.

A "better" programming language will not help you become a better programmer. It is not about the language, it is the mindset that makes a good programmer. The ability to understand why something does not work as intended and then fix it, is what counts.

For me plain C without libraries and HAL work very well on bare metal embedded projects. Yes you have to do everything yourself, but over time you build up your own set of code that can be used over and over again. The benefit of this is that you know what everything does.

I have seen things go sideways with for example C#, where a lot is done for you, and it can take a lot of effort to fix it when it turns out to be a bug in the libraries that come with it. Also had to work around bugs in javascript that can take a lot of time to figure out why things go wrong. Even with C++ and then perform bound checks, I have seen memory leaks within the libraries.

But like others wrote, if you like doing something like this go ahead, but I expect that not a lot of the expert embedded developers want to become testers of your new language.

What - specifically - do you find problematic with C# ? do you have an actual, real world example you can share?

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 10:03:58 pm
Most of the complaining here is not about specific suggestions about language improvements or changes, but more about the fact that someone is actually daring to rock the boat and discuss the subject dispassionately, point out the elephant in the room!
Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 25, 2022, 10:32:48 pm
Most of the complaining here is not about specific suggestions about language improvements or changes,
Once again, this is not the case. All your suggestions were addressed and discussed.

daring to rock the boat
You would rock the boat if you come up even with a prototype compiler or coherent description of the proposed language. You are not the first one to want a new better language. Some people did the work and ended up with Zig and Rust and a ton of others. If you don't like them, propose what you would change. But if all you would change is a syntax for the computed gotos, then don't blame us that we don't care.

Do the work, propose something concrete to discuss and we will gladly do so. Right now all we have is a set of random ideas.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 25, 2022, 10:36:51 pm
Most of the complaining here is not about specific suggestions about language improvements or changes, but more about the fact that someone is actually daring to rock the boat and discuss the subject dispassionately, point out the elephant in the room!

Not quite.

It is pointing out that if someone suggests jumping out of the frying pan, they should have good reasons why they aren't jumping into the fire. Or some similar tortured analogy!
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 10:56:36 pm
Most of the complaining here is not about specific suggestions about language improvements or changes,
Once again, this is not the case. All your suggestions were addressed and discussed.

daring to rock the boat
You would rock the boat if you come up even with a prototype compiler or coherent description of the proposed language. You are not the first one to want a new better language. Some people did the work and ended up with Zig and Rust and a ton of others. If you don't like them, propose what you would change. But if all you would change is a syntax for the computed gotos, then don't blame us that we don't care.

Do the work, propose something concrete to discuss and we will gladly do so. Right now all we have is a set of random ideas.

You're advised to reread the entire thread if you are under the illusion nothing concrete has been proposed, that's probably why you're confused.

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 10:59:48 pm
Most of the complaining here is not about specific suggestions about language improvements or changes, but more about the fact that someone is actually daring to rock the boat and discuss the subject dispassionately, point out the elephant in the room!

Not quite.

It is pointing out that if someone suggests jumping out of the frying pan, they should have good reasons why they aren't jumping into the fire. Or some similar tortured analogy!

Sorry to point out, but the thread is not about frying pans or jumping, perhaps you're thinking of something else? This is what's known in debating circles as a strawman argument, rather than actually debate some specific thing I might have actually proposed or suggested you prefer to attack an imaginary argument, disagreeing with something I never even said.

I thought I was dealing with engineers, dispassionate logical thinkers, I'm seeing rather a lot of emotion and confusuion.

Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 25, 2022, 11:06:57 pm
Everything you proposed here is trivial and was already known, none of this is enough to justify making a new language or even extending the exiting ones.

Most people here are engineers, and everyone participating in this thread expressed their opinion on your proposals. This is the engineering opining based on people's experience with using the exiting tools. And the consensus seems to be that none of this is really necessary. May be nice to have if it was already there, but not necessary. What else is to discuss here?

If you want more safety and better defined behaviour, use Rust. Many companies and individuals are switching to it instead of C.

But if you want to think that we all are wrong, then there is nothing we can do about it.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 11:16:34 pm
Everything you proposed here is trivial and was already known, none of this is enough to justify making a new language or even extending the exiting ones.

I disagree.

Most people here are engineers, and everyone participating in this thread expressed their opinion on your proposals. This is the engineering opining based on people's experience with using the exiting tools. And the consensus seems to be that none of this is really necessary. May be nice to have if it was already there, but not necessary. What else is to discuss here?

Even if this were true it matters not, this is an informal discussion about ways one might produce a new programming language for MCU use. That you disapprove is irrelevant, your opinions are noted, why persist in these perpetual complaints? why do you care?

If you want more safety and better defined behaviour, use Rust. Many companies and individuals are switching to it instead of C.

As I suspected you are confused, please show us all the post where I said I wanted "more safety and better defined behavior" can you do that? No, because it is false statement, there is no such post, you either imagined it or made it up. I do hope you code better than you troll.

Title: Re: A new, hardware "oriented" programming language
Post by: ataradov on November 25, 2022, 11:22:43 pm
this is an informal discussion about ways one might produce a programming language for MCU use.
At this thread shows, no one here is interested in a language specifically for MCUs. Why would we? What if your next project needs and MPU as well (as increasingly happens)? Whys would we not what to use the same exact tool set?

"Elephant in the room" here is that we don't see a point in discussing this topic. Not because we are entrenched with C and refuse to even look at something else. Most of us looked at other stuff a lot, and in the end C ends up being the best option when you consider everything, not just a very narrow use case of programming low level PICs.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 25, 2022, 11:28:27 pm
this is an informal discussion about ways one might produce a programming language for MCU use.
At this thread shows, no one here is interested in a language specifically for MCUs. Why would we? What if your next project needs and MPU as well (as increasingly happens)? Whys would we not what to use the same exact tool set?

If you believe that then why not just ignore the thread instead of clogging it with your pessimism and vitriol, do you really have nothing better to do with your time?

"Elephant in the room" here is that we don't see a point in discussing this topic. Not because we are entrenched with C and refuse to even look at something else. Most of us looked at other stuff a lot, and in the end C ends up being the best option when you consider everything, not just a very narrow use case of programming low level PICs.

We're done young man, I've been working with electronics, radio and computers since before you were born, you're doing a disservice to your employer Microchip with this intolerant (dare I say, arrogant) and rude behavior in a public forum, you are - like it or not - a representative of Microchip and at this point I am far from impressed with your manner, you can whine and winge about me all you like, I have nothing more to say to you, good evening.

Now, getting back to the actual subject...



Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 25, 2022, 11:58:35 pm
Most of the complaining here is not about specific suggestions about language improvements or changes, but more about the fact that someone is actually daring to rock the boat and discuss the subject dispassionately, point out the elephant in the room!

Bullshit.

It's frequently my job to "rock the boat" by proposing changes and improvements to processes, libraries, programming languages, and even new machine code instructions. You can find my name for example in the credits for the RISC-V base instruction set and the B and V extensions.

Your proposed changes address ONE aspect of the problem:

- does the change make a programmer's life slightly more convenient?

You totally ignore every other aspect, such as:

- can the proposed feature be efficiently implemented on the target devices?

- is it a significant improvement vs using a library (function, macro) or maybe code generator instead?

- is the product of the improvement and the frequency of use sufficient to justify the incremental cumulative increase in complexity of the language, compiler, manuals, training?
Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 26, 2022, 02:04:11 am
There is no surprise that there are engineers quite content with C, or even committed to C.
When talking about low level programming languages –– which is what I understand 'a hardware "oriented" programming language' to mean ––, C is just the one with the best proven track record decades long.  It isn't that great, it's just the 'benchmark' one for others, due to its widespread use and role in systems programming and embedded development.

For examples of nearly bug-free programs written in C in the systems programming domain, go check out D. J. Bernstein (https://cr.yp.to/djb.html)'s djbdns (https://cr.yp.to/djbdns.html), qmail (https://cr.yp.to/qmail.html), daemontools (https://cr.yp.to/daemontools.html), cdb (https://cr.yp.to/cdb.html).  This is the guy behind Curve25519 (https://en.wikipedia.org/wiki/Curve25519), having released it in 2005.

Like it, dislike it, doesn't matter, C is just a tool.  But as a tool, its features and track record are significant.  So are its deficiencies, and that means any real effort to do better is valuable.

In comparison, C# is a managed language. .NET Micro requires at least 256k of RAM. .NET nanoFramework requires at least 64k of RAM, and runs on Cortex-M and RISC-V (ESP32-C3) cores.  So, perhaps suitable for medium to large embedded devices, but decidedly unsuitable for small ARMs and anything less than 32-bit architecures.

Ada can be used to program AVR 8-bit microcontrollers (see AVR-Ada), but it is still relatively little used.  One possible reason is that while GCC GNAT is GPL3+ licensed with a runtime library exception, AdaCore sells GNAT Pro, and the FSF/GCC GNAT is seen as "inferior", with the "proper" version being the sole product of a commercial company.  (Or maybe that's just me.)

I get that some consider this pointless
No, that's not it at all.  Not pointless, more like bass-ackwards.  We want the results too, we just have seen your approach before leading to nowhere.  We're trying to steer you to not repeat that, but actually produce something interesting.

If you start a language design from scratch, you must understand the amount of design choices already made for existing languages.  The ones in languages that have survived use in anger, are the ones where the choices support a programming paradigm the users find intuitive and effective.

Why did DiTBHo not start from scratch, and instead pared down C to a subset with some changes and additions, to arrive at their my-C, designed for strictly controlled and enforced embedded use cases?  Because they needed a tool fit for a purpose, and it was a straightforward way to achieve it.  Results matter.

Why did SiliconWizard's Design a better "C" (https://www.eevblog.com/forum/programming/design-a-better-c/) thread 'not go anywhere'?  It just sprawled around, with individual features and other languages discussed.  In fact, it really showed how complicated and hard it is to do better than C from scratch; with other languages like Ada (https://en.wikipedia.org/wiki/Ada_(programming_language)) discussed but nobody knowing exactly why they never got as much traction as C.  Just consider this post by brucehoult (https://www.eevblog.com/forum/programming/design-a-better-c/msg3618621/#msg3618621) about midway in the thread, about how C with its warts and all still maps to different hardware so well.

Me, I have worked on replacing the standard C library with something better.  Because the C standard defines freestanding environment where the C standard library is not available in quite a detail –– unlike say C++, which also has the same concept, but leaves it basically completely up to implementations to define what it means ––, this is doable.  I aim to fix many of the issues others have with C.  With C23 around the corner, the one change I think might actually make a difference is to arrays not decay to pointers, and instead conceptually use arrays everywhere to describe memory ranges.  Even just allowing type variation based on a later variable in the same argument list would make it possible to replace buffer overrun prone standard library functions with almost identical replacements, that would allow the C compiler to detect buffer under- and overruns at compile time.  It would only be a small addition, perhaps a builtin, to make it possible to prove via static analysis that all memory accesses are valid.
In other words, I'm looking to change the parts of C that hinder me or others, not start from scratch.

Am I a C fanboi?  No.  If you look at my posting history, you'll see that I actually recommend using an interpreted language, currently Python, for user interfaces (for multiple reasons).

I currently use C for some embedded (AVRs, mainly), and a mixed C/C++ freestanding environment for embedded ARM development; I also use POSIX C for systems programming in Linux (mostly on x86-64).  (I sometimes do secure programming, dealing with privileges and capabilities; got some experience as a sysadmin at a couple of universities, and making customized access solutions for e.g. when you have a playground with many users at different privilege levels, and subsections with their own admins, including sub-websites open to the internet.  It's not simple when you're responsible nothing leaks that shouldn't leak.)

Okay, so if we believe that a ground-up design from scratch is unlikely to lead to an actual project solving the underlying problems OP (Sherlock Holmes) wants to solve, what would?

Pick a language, and a compiler, you feel you can work with.  It could be C, it could be Ada, it could be whatever you want.  Obviously, it should have somewhat close syntax to what you prefer, but it doesn't have to be an exact match.  I'll use C as the language example below for simplicity only; feel free to substitute it with something else.

Pick a problem, find languages that solve it better than C, or invent your own new solution.  Trace it down to the generated machine code, and find a way to port it back to C, replacing the way C currently solves it.  Apply it in real life, writing real-world code that heavily uses that modified feature.  Get other people to comment on it, and maybe even test it.  Find out if the replacement solution actually helps with real-world code.  That often means getting an unsuspecting victim, and having them re-solve a problem using the modified feature, using only your documentation of the feature as a guide.

Keep a journal of your findings.

At some point, you find that you have enough of those new solutions to construct a completely new language.  At this point, you can tweak the syntax to be more to your liking.  Start writing your own compiler, but also document the language the compiler works with, precisely.  As usual, something like ABNF is sufficient for syntax, but for the paradigm, the approach, I suggest writing additional documentation explaining your earlier findings, and the solution approach.  Small examples are gold here.  The idea is that other people, reading this additional documentation, can see how you thought, so they can orient themselves to best use the new language.

Theory is nice, but practical reality always trumps theory.  Just because the ABNF of a language looks nice, doesn't mean it is an effective language.  As soon as you can compile running native binaries, start creating actual utilities – sort, grep, bc for example –, and look at the machine code the compiler generates.  Just because the code is nice and the abstractions just perfect, does not mean they are fit for generating machine code.  Compare the machine code to what the original language and other languages produce, when optimizations are disabled (for a more sensible comparison).

During this process, do feel free to occasionally branch into designing your language from scratch.  If you keep tabs on your designs as your understanding evolves, you'll understand viscerally what the 'amount of design choices' I wrote above really means.  It can be overwhelming, if you think of it, but going at it systematically, piece by piece, with each design choice having an explanation/justification in your journal and/or documentation, it can be done, and done better than what we have now.

Finally: I for one prefer passionate, honest, detailed posts over dispassionate politically correct smooth-talk.
Title: Re: A new, hardware "oriented" programming language
Post by: Wilksey on November 26, 2022, 02:46:49 am
When I first saw the title I thought one was referring to an alternative to VHDL / Verilog.
Growing up in the 80's I was a committed user of 6502 and z80 assembler having owned Acorn and Amstrad machines, the BASIC language set was just not cutting it for me.  Cue the 90's and PC's became more available, we still had BASIC (in the form of MS QBASIC and Visual Basic) and the new fangled world of C/C++ (Borland, Microsoft, Watcom etc) and Pascal, and of course, assembler, sure these probably existed before then but in my neck of the woods it wasn't as widely discussed or represented in the likes of magazines until then.

Years later (cut to today) there are many languages available for the PC but chips like microcontrollers and microprocessors still have a few options, BASIC, Pascal or C/C++, even assembler is being less favoured by modern toolchains.

I would be interested to see what a "new" language would look like and what the uptake would be, I think C/C++ has just become the standard target tool when manufacturers bring out a new chip.

It is an interesting idea, and this must have been tried and tested before so why did it not take off, and what would be different to your offering vs a previous failed one.
I don't know if it would gain enough traction to disrupt the current market as C is supported and understood on most if not all microprocessors / microcontrollers.
Title: Re: A new, hardware "oriented" programming language
Post by: pcprogrammer on November 26, 2022, 06:08:28 am
What - specifically - do you find problematic with C# ? do you have an actual, real world example you can share?

I have left C# behind me some 20 years ago, basically not long after it came up. Maybe it has improved over time, but for the work I did and do I never needed or wanted to use it. Did work for a company that changed over to it for their front end of the product and they had a hard time getting it to work properly, but that is also long ago.

My origin in programming started with basic and went onto Z80 assembler soon after. Then 8051 and 6502 assembler. Loved it very much because you are in control all the way. Started with C in 1996 and stuck with it ever since, apart from projects where C++, ASP, PHP, javascript or basicscript where needed.

I don't feel the need for a language that protects me from making mistakes.

Edit:

With embedded work it depends on the microcontroller one is using what is useful. With only 20KB of RAM, some memory management can become a pain in the bum, and for most tasks it is known what the needed memory is, and will be setup during programming. No dynamic allocating needed, no garbage collection that often fails, and so on.

When diving into the Internet Of Things this becomes more in play, but then you start to need beefier microcontrollers with more memory to get things done. And sure for a lot of programmers it will be beneficial to have a language that does all the hard work for them, but when shit hits the fan, they will have an even harder time to solve the problem. That is why I, and probably ataradov, like C so much. If you don't want it, you can keep all the "luxury" out the door.

For instance, with the Arduino setup you can write up something quickly and have it work, but the moment you need something special it becomes harder to realize it, because of the lack of transparency. And yes I have looked under the hood of the Arduino system and it is diving into a lot of sub directories and header files to find something specific. Same with the STM32 HAL code. In the end I find it easier to just read the reference manual of a microcontroller and write code for it at the lowest level possible without resorting to assembler.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 09:11:25 am
Most of the complaining here is not about specific suggestions about language improvements or changes, but more about the fact that someone is actually daring to rock the boat and discuss the subject dispassionately, point out the elephant in the room!

Bullshit.

It's frequently my job to "rock the boat" by proposing changes and improvements to processes, libraries, programming languages, and even new machine code instructions. You can find my name for example in the credits for the RISC-V base instruction set and the B and V extensions.

Yup; my background was similar.

Quote
Your proposed changes address ONE aspect of the problem:

- does the change make a programmer's life slightly more convenient?

You totally ignore every other aspect, such as:

- can the proposed feature be efficiently implemented on the target devices?

That's the one that should kill most language proposals. Too often it doesn't, and the result is Domain Specific Languages rather than Domain Specific Libraries.

DSLanguages almost always grow until they become tangled and incomprehensible. They also lack tool support, which is a vital ecosystem component of widely used languages.

Quote
- is it a significant improvement vs using a library (function, macro) or maybe code generator instead?

- is the product of the improvement and the frequency of use sufficient to justify the incremental cumulative increase in complexity of the language, compiler, manuals, training?

Spot on.

Don't forget that it is difficult to employ staff to work on anything which won't help them to get another job.

However, creating your own language is fun, just as creating your own processor/ISA is fun. But don't expect other people to want to use it!
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 09:26:04 am
When diving into the Internet Of Things this becomes more in play, but then you start to need beefier microcontrollers with more memory to get things done. And sure for a lot of programmers it will be beneficial to have a language that does all the hard work for them, but when shit hits the fan, they will have an even harder time to solve the problem. That is why I, and probably ataradov, like C so much. If you don't want it, you can keep all the "luxury" out the door.

For instance, with the Arduino setup you can write up something quickly and have it work, but the moment you need something special it becomes harder to realize it, because of the lack of transparency. And yes I have looked under the hood of the Arduino system and it is diving into a lot of sub directories and header files to find something specific. Same with the STM32 HAL code. In the end I find it easier to just read the reference manual of a microcontroller and write code for it at the lowest level possible without resorting to assembler.

Yup.

I used a naked "8-pin Arduino" in my Vetinari clock (https://entertaininghacks.wordpress.com/2015/02/23/vetinari-digital-clock/), because that was the quick way to get an undemanding result.

But when making something that required very low power consumption, I simply wrote C to peek/poke the atmega328 registers. That avoided me having to work out how Arduino libraries might be frustrating me.
Title: Re: A new, hardware "oriented" programming language
Post by: iMo on November 26, 2022, 09:47:07 am
A need for a hw-oriented language? There is the Forth  :D
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 26, 2022, 10:02:32 am
Why did SiliconWizard's Design a better "C" (https://www.eevblog.com/forum/programming/design-a-better-c/) thread 'not go anywhere'?  It just sprawled around, with individual features and other languages discussed.  In fact, it really showed how complicated and hard it is to do better than C from scratch; with other languages like Ada (https://en.wikipedia.org/wiki/Ada_(programming_language)) discussed but nobody knowing exactly why they never got as much traction as C.  Just consider this post by brucehoult (https://www.eevblog.com/forum/programming/design-a-better-c/msg3618621/#msg3618621) about midway in the thread, about how C with its warts and all still maps to different hardware so well.

I note that one of the desires there was "give easy access to the flags". I pointed out that many ISAs don't even *have* flags. A lot more people in this group have been gaining experience with such CPUs in the 16 months since that post :-) But even on ISAs that have flags, there are big differences. The DEC/Motorola/ARM group have a family similarity, with differences, but the Intel family is really quite different.

Access to the NZ flags (SZ on Intel) is trivial, so when people say they want access to the flags what they usually mean is carry (C,CY), sometimes overflow (V,OF), rarely parity or nibble carry (both on Intel).

Even for the people who just want Carry, I highly doubt they are prepared for the different implementations of the carry flag with respect to subtraction and compare.

And it's simply not hard to express what you ACTUALLY WANT in portable C anyway. The compiler will automatically use the flags where available.

Code: [Select]
unsigned long add_carry(unsigned long x, unsigned long y, unsigned long a, unsigned long b){
    return (a+b) < a ? x+y+1 : x+y;
}

x86:

Code: [Select]
        mov     rax, rdi
        add     rdx, rcx
        adc     rax, rsi
        ret

Exactly what you'd hope for.

ARMv7:

Code: [Select]
        cmn     r2, r3
        it      cs
        addcs   r1, r1, #1
        add     r0, r0, r1
        bx      lr

The code could be better, actually, but at least branch-free.

ARMv8:

Code: [Select]
        cmn     x2, x3
        adc     x0, x1, x0
        ret

Fine.

RISC-V:

Code: [Select]
        not     a2, a2
        sltu    a2, a2, a3
        add     a0, a0, a1
        add     a0, a0, a2
        ret

No flags needed. Branch-free too.

Interesting that both ARM versions and RISC-V all chose to use "compare with negative" instead of adding, but that works too.


Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 10:10:05 am
A need for a hw-oriented language? There is the Forth  :D

Once, 40 years ago, I had a couple of weeks to bring up a new peripheral created by someone else. In order to reduce the amount of typing, I implemented a quick and very dirty macro expansion tool. It grew a bit, was sufficient for the job, but the limitations were already becoming apparent. Fortunately it could be thrown away after the job ended, and was.

I learned two things, which have stood the test of time:

I've seen that first lesson repeated (not by me!) several times over the succeeding decades.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 10:12:19 am
Even for the people who just want Carry, I highly doubt they are prepared for the different implementations of the carry flag with respect to subtraction and compare.

And it's simply not hard to express what you ACTUALLY WANT in portable C anyway. The compiler will automatically use the flags where available.

Bignums aren't easy in portable C :)
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 26, 2022, 12:09:33 pm
Even for the people who just want Carry, I highly doubt they are prepared for the different implementations of the carry flag with respect to subtraction and compare.

And it's simply not hard to express what you ACTUALLY WANT in portable C anyway. The compiler will automatically use the flags where available.

Bignums aren't easy in portable C :)

Well, yes they are, as just demonstrated.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 02:22:53 pm
Most of the complaining here is not about specific suggestions about language improvements or changes, but more about the fact that someone is actually daring to rock the boat and discuss the subject dispassionately, point out the elephant in the room!

Bullshit.

It's frequently my job to "rock the boat" by proposing changes and improvements to processes, libraries, programming languages, and even new machine code instructions. You can find my name for example in the credits for the RISC-V base instruction set and the B and V extensions.

Your proposed changes address ONE aspect of the problem:

- does the change make a programmer's life slightly more convenient?

You totally ignore every other aspect, such as:

- can the proposed feature be efficiently implemented on the target devices?

- is it a significant improvement vs using a library (function, macro) or maybe code generator instead?

- is the product of the improvement and the frequency of use sufficient to justify the incremental cumulative increase in complexity of the language, compiler, manuals, training?

No, take a good look, you'll see my remark has some basis.

For an engineer, you're not very thorough or objective, you'll find no suggestion from me about "convenience", you're paraphrasing.

You've also not asked me a single specific question about any feature on any target device, furthermore at this stage I've been specifically focusing on grammar not semantics,  I said this several times too.

Your position is pessimistic, discouraging, not open minded, this is nothing to do with your undoubted credentials either, it is you, your hostile tone. This is just a discussion and you're frothing and fuming.

Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 26, 2022, 03:02:10 pm
I used a naked "8-pin Arduino" in my Vetinari clock (https://entertaininghacks.wordpress.com/2015/02/23/vetinari-digital-clock/), because that was the quick way to get an undemanding result.

But when making something that required very low power consumption, I simply wrote C to peek/poke the atmega328 registers. That avoided me having to work out how Arduino libraries might be frustrating me.
That's exactly why ATmega32u4 is one of my favourite 8-bitters.  It has native full-speed USB (12 Mbit/s, can do 1 Mbyte/sec even over USB Serial), but still is completely programmable without any HAL or similar, on bare metal, using nothing but avr-gcc (and avr-libc, if you really want), in (freestanding) C or C++.

I note that one of the desires there was "give easy access to the flags". I pointed out that many ISAs don't even *have* flags.
Yep.  In particular, you helped me realize I did not actually desire access to flags, just multiple function result values, to fulfill my needs.

An extremely common pattern for functions is to return a value or error.  Obviously, with scalar return types, when you return a single scalar, some values must be reserved for error codes.  (That can get messy, like it once did in the Linux kernel, where certain filesystem drivers could not handle reads over 231 bytes (due to widespread use of int), and returned negative values.  Now, the Linux kernel limits all single read() and write() syscalls to 231 less one page automagically.)  Having a separate value and status returns is sometimes more robust, even if the status return is just a single "OK"/"Error" flag.

However, SysV ABI on x86-64 (https://wiki.osdev.org/System_V_ABI#x86-64) (LP64, so ints are 32-bit, long and pointers are 64-bit) as used on Linux, macOS, FreeBSD, and others, already supports this (by using rax and rdx registers for 128-bit non-FP return values), so code like
    typedef struct { long x, y } ipair;
    lpair somefunc(long x, long y) { return (lpair){ .x = x, .y = y }; }
compiles to
    somefunc:
        mov  rax, rdi
        mov  rdx, rsi
        ret
because rdi and rsi are the first two (non-FP) parameters, and the pair of return values are returned in rax and rdx.
This suffices for my needs, although it is slightly odd-looking at first, wrapping return values into structs.  It would be nicer to have syntactic sugar for this and not require struct use, but this works too.

Because this is architecture ABI, any compiler for this architecture can use this.  Other ABIs, however, need to do annoying "emulation" by e.g. passing a pointer to the additional return values, similar to how larger (struct-type) result values are passed.  A compiler can obviously use its own function call ABI internally, but it does have to have some kind of foreign function interface* for bindings to binary libraries (if used on hosted environments/full OSes, and not just bare metal embedded).  It might be useful to use a rarely used register for the status/error return, so that it is not so often clobbered by ordinary code.

Simply put, I wasn't hankering so much for syntactic sugar, but for more sensible ABIs like SysV on x86-64.  Brucehoult put me straight on this, albeit indirectly.  :-+



* Part of the reason I like using Python for graphical user interfaces is that it has a very nice interface to calling native code, without needing any native code support; using pure Python code for the interface.  Certain libraries, like glib's gobject introspection for C, provide introspection information (.gi or gir files) that means full interfaces can be constructed without any bespoke code (machine or Python), using the introspection information only.  It also means that when the library and corresponding introspection file is updated, the new features are immediately available in Python also.  Very useful.

Because native machine code is dynamically loaded for access from Python, this also provides a perfect mechanism for license separation.  You can keep the user interface open source, modifiable by end users, but keep all Sekret Sauce proprietary code in your binary-only dynamic library.  In my experience, this provides the optimal balance between user access and proprietary requirements when going full open source isn't viable, e.g. for commercial reasons.
Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 26, 2022, 03:04:57 pm
Most of the complaining here is not about specific suggestions about language improvements or changes, but more about the fact that someone is actually daring to rock the boat and discuss the subject dispassionately, point out the elephant in the room!

Bullshit.

It's frequently my job to "rock the boat" by proposing changes and improvements to processes, libraries, programming languages, and even new machine code instructions. You can find my name for example in the credits for the RISC-V base instruction set and the B and V extensions.

Your position is pessimistic, discouraging, not open minded, this is nothing to do with your undoubted credentials either, it is you, your hostile tone. This is just a discussion and you're frothing and fuming.
Oh come on, now you're just sounding like a petulant child that is disappointed that someone is pointing out a flaw in their Perfect Plan yourself; one that is expecting adulation and praise.  Get your ego out of the discussion, and concentrate on the subject matter instead.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 03:45:20 pm
Most of the complaining here is not about specific suggestions about language improvements or changes, but more about the fact that someone is actually daring to rock the boat and discuss the subject dispassionately, point out the elephant in the room!

Bullshit.

It's frequently my job to "rock the boat" by proposing changes and improvements to processes, libraries, programming languages, and even new machine code instructions. You can find my name for example in the credits for the RISC-V base instruction set and the B and V extensions.

Your position is pessimistic, discouraging, not open minded, this is nothing to do with your undoubted credentials either, it is you, your hostile tone. This is just a discussion and you're frothing and fuming.
Oh come on, now you're just sounding like a petulant child that is disappointed that someone is pointing out a flaw in their Perfect Plan yourself; one that is expecting adulation and praise.  Get your ego out of the discussion, and concentrate on the subject matter instead.

So, what specifically did you want to say about language grammar, extensibility, features that could be helpful to MCU developers? Simply bleating (as some are) that "there's nothing wrong with C, I'm satisfied with C, discussing always leads nowhere" is pointless, anyone who's not the slightest bit interested in possibilities can just ignore the thread, isn't that reasonable?
Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 26, 2022, 04:11:09 pm
Bignums aren't easy in portable C :)
If your limbs are of uint32_t type, you can use uint_fast64_t for the intermediate results.  On architectures without a 32×32=64-bit unsigned multiplication or 64/32=32+32 unsigned division with remainder, the machine code will be pretty inefficient, but it will work.

uint32_t, uint_fast64_t et cetera are declared in <stdint.h>, which is provided by the compiler (since C99, so for over two decades now), and is available even in freestanding environments where the standard C library is not available.

So, they ackshually are.  Efficient bignums, on the other hand...

For binary (integer, fixed point, floating point) to decimal (string) conversions and vice versa, on 32-bit architectures (embedded ARMs), I like to use something I call m28, an array of uint32_t's where the array index zero is just next to the decimal point, and grows away from the decimal point; and each 32-bit unsigned integer contains 28 bits of the composite bignum.  (m28i is for integer parts, least significant limb first; and m28f for fractional parts, most significant limb first.)

This means that multiplying a bignum by an integer between 2 and 15, inclusive, only requires one 32×32=32-bit multiplication per limb; and dividing a bignum by an integer between 2 and 15, inclusive, only one 32-bit division with remainder, or 32-bit division without remainder and a multiplication, per limb.
Addition and subtraction of bignums does need an AND mask (by 0x0FFFFFFF) and a 28-bit right shift and two additions/subtractions per limb (due to carry), or an addition/subtraction and a conditional increment/decrement.

This is amazingly efficient.  (I just need to find out how to avoid having to have an array of all powers of ten precomputed, to implement float/double conversion support with strict upper limit on the memory use during conversion, with exact correct LSB rounding in place.  With floats, it's only 83 constants, requiring max. 2988 bytes of Flash/ROM, so maybe it would be acceptable.. Undecided.)

In essence, I'm spending 14.3% of memory extra to make the computation simpler and more effective for single digit octal/decimal/hexadecimal extraction/insertion, but it turns out it's worth it on typical 32-bit architectures (ARM cores, RISC-V, using Compiler Explorer/godbolt.org to experiment with).
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 04:19:01 pm
So, what specifically did you want to say about language grammar, extensibility, features that could be helpful to MCU developers?

There is very little (if anything) about language grammar that could be useful to MCU developers.

If extensibility is important, then domain specific libraries are the way to go.

Your initial questions were about far more than grammar, and therefore potentially more interesting. That's why people contributed.

You have consistently failed to understand that the main topic of importance is the runtime behaviour (in a broad sense) of something expressed in a language. Anything that improves the runtime behaviour is potentially important.
Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 26, 2022, 04:27:32 pm
So, what specifically did you want to say about language grammar, extensibility, features that could be helpful to MCU developers?
I'm saying that they are not at all important to microcontroller or embedded developers at all.

I am saying that you are trying to solve a problem by daydreaming about irrelevant stuff, when you could be working on making things actually better.

Are you sure you're not a Windows C# enthusiast who dreams of creating a C#-like language that makes all embedded devices look like Windows or at least .Net runtime –– because "Windows is basically the entire world, after all", and being adulated for it?  You definitely sound like one.  I've met many, and all have either crashed or burned, or keep making unreliable unsecure shit and getting by with their social skills alone.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 05:38:04 pm
There is no surprise that there are engineers quite content with C, or even committed to C.

When talking about low level programming languages –– which is what I understand 'a hardware "oriented" programming language' to mean ––, C is just the one with the best proven track record decades long.  It isn't that great, it's just the 'benchmark' one for others, due to its widespread use and role in systems programming and embedded development.

For examples of nearly bug-free programs written in C in the systems programming domain, go check out D. J. Bernstein ([url]https://cr.yp.to/djb.html[/url])'s djbdns ([url]https://cr.yp.to/djbdns.html[/url]), qmail ([url]https://cr.yp.to/qmail.html[/url]), daemontools ([url]https://cr.yp.to/daemontools.html[/url]), cdb ([url]https://cr.yp.to/cdb.html[/url]).  This is the guy behind Curve25519 ([url]https://en.wikipedia.org/wiki/Curve25519[/url]), having released it in 2005.

Like it, dislike it, doesn't matter, C is just a tool.  But as a tool, its features and track record are significant.  So are its deficiencies, and that means any real effort to do better is valuable.

In comparison, C# is a managed language. .NET Micro requires at least 256k of RAM. .NET nanoFramework requires at least 64k of RAM, and runs on Cortex-M and RISC-V (ESP32-C3) cores.  So, perhaps suitable for medium to large embedded devices, but decidedly unsuitable for small ARMs and anything less than 32-bit architecures.

Ada can be used to program AVR 8-bit microcontrollers (see AVR-Ada), but it is still relatively little used.  One possible reason is that while GCC GNAT is GPL3+ licensed with a runtime library exception, AdaCore sells GNAT Pro, and the FSF/GCC GNAT is seen as "inferior", with the "proper" version being the sole product of a commercial company.  (Or maybe that's just me.)

I get that some consider this pointless

No, that's not it at all.  Not pointless, more like bass-ackwards.  We want the results too, we just have seen your approach before leading to nowhere.  We're trying to steer you to not repeat that, but actually produce something interesting.

If you start a language design from scratch, you must understand the amount of design choices already made for existing languages.  The ones in languages that have survived use in anger, are the ones where the choices support a programming paradigm the users find intuitive and effective.

Why did DiTBHo not start from scratch, and instead pared down C to a subset with some changes and additions, to arrive at their my-C, designed for strictly controlled and enforced embedded use cases?  Because they needed a tool fit for a purpose, and it was a straightforward way to achieve it.  Results matter.

Why did SiliconWizard's Design a better "C" ([url]https://www.eevblog.com/forum/programming/design-a-better-c/[/url]) thread 'not go anywhere'?  It just sprawled around, with individual features and other languages discussed.  In fact, it really showed how complicated and hard it is to do better than C from scratch; with other languages like Ada ([url]https://en.wikipedia.org/wiki/Ada_(programming_language))[/url] discussed but nobody knowing exactly why they never got as much traction as C.  Just consider this post by brucehoult ([url]https://www.eevblog.com/forum/programming/design-a-better-c/msg3618621/#msg3618621[/url]) about midway in the thread, about how C with its warts and all still maps to different hardware so well.

Me, I have worked on replacing the standard C library with something better.  Because the C standard defines freestanding environment where the C standard library is not available in quite a detail –– unlike say C++, which also has the same concept, but leaves it basically completely up to implementations to define what it means ––, this is doable.  I aim to fix many of the issues others have with C.  With C23 around the corner, the one change I think might actually make a difference is to arrays not decay to pointers, and instead conceptually use arrays everywhere to describe memory ranges.  Even just allowing type variation based on a later variable in the same argument list would make it possible to replace buffer overrun prone standard library functions with almost identical replacements, that would allow the C compiler to detect buffer under- and overruns at compile time.  It would only be a small addition, perhaps a builtin, to make it possible to prove via static analysis that all memory accesses are valid.
In other words, I'm looking to change the parts of C that hinder me or others, not start from scratch.

Am I a C fanboi?  No.  If you look at my posting history, you'll see that I actually recommend using an interpreted language, currently Python, for user interfaces (for multiple reasons).

I currently use C for some embedded (AVRs, mainly), and a mixed C/C++ freestanding environment for embedded ARM development; I also use POSIX C for systems programming in Linux (mostly on x86-64).  (I sometimes do secure programming, dealing with privileges and capabilities; got some experience as a sysadmin at a couple of universities, and making customized access solutions for e.g. when you have a playground with many users at different privilege levels, and subsections with their own admins, including sub-websites open to the internet.  It's not simple when you're responsible nothing leaks that shouldn't leak.)

Okay, so if we believe that a ground-up design from scratch is unlikely to lead to an actual project solving the underlying problems OP (Sherlock Holmes) wants to solve, what would?

Pick a language, and a compiler, you feel you can work with.  It could be C, it could be Ada, it could be whatever you want.  Obviously, it should have somewhat close syntax to what you prefer, but it doesn't have to be an exact match.  I'll use C as the language example below for simplicity only; feel free to substitute it with something else.

Pick a problem, find languages that solve it better than C, or invent your own new solution.  Trace it down to the generated machine code, and find a way to port it back to C, replacing the way C currently solves it.  Apply it in real life, writing real-world code that heavily uses that modified feature.  Get other people to comment on it, and maybe even test it.  Find out if the replacement solution actually helps with real-world code.  That often means getting an unsuspecting victim, and having them re-solve a problem using the modified feature, using only your documentation of the feature as a guide.

Keep a journal of your findings.

At some point, you find that you have enough of those new solutions to construct a completely new language.  At this point, you can tweak the syntax to be more to your liking.  Start writing your own compiler, but also document the language the compiler works with, precisely.  As usual, something like ABNF is sufficient for syntax, but for the paradigm, the approach, I suggest writing additional documentation explaining your earlier findings, and the solution approach.  Small examples are gold here.  The idea is that other people, reading this additional documentation, can see how you thought, so they can orient themselves to best use the new language.

Theory is nice, but practical reality always trumps theory.  Just because the ABNF of a language looks nice, doesn't mean it is an effective language.  As soon as you can compile running native binaries, start creating actual utilities – sort, grep, bc for example –, and look at the machine code the compiler generates.  Just because the code is nice and the abstractions just perfect, does not mean they are fit for generating machine code.  Compare the machine code to what the original language and other languages produce, when optimizations are disabled (for a more sensible comparison).

During this process, do feel free to occasionally branch into designing your language from scratch.  If you keep tabs on your designs as your understanding evolves, you'll understand viscerally what the 'amount of design choices' I wrote above really means.  It can be overwhelming, if you think of it, but going at it systematically, piece by piece, with each design choice having an explanation/justification in your journal and/or documentation, it can be done, and done better than what we have now.

Finally: I for one prefer passionate, honest, detailed posts over dispassionate politically correct smooth-talk.


That's a very good post, thank you for spending the time to articulate that.

I don't really have an "approach" here, this thread was expected to just be a good natured chin wag about languages, like, dislikes, aspirations with - ideally - constructive input from professional MCU developers, input about what they'd like to see done better in a language or what things they can't do that they'd like to be able to do, so that was where I'd expected this thread to go.

I have used C professionally for decades, I used to work at the Financial Times and the London Stock Exchange, designing data feed handlers for umpteen protocols and data formats (X.25, async, IBM Bisync, TCP etc) in the 1980s on Stratus hardware fault tolerant computers, I lead teams, I was a lead designer.

I've also designed and developed high performance shared memory data storage technology for Windows NT with a concurrent heap that can support multiple processes each with multiple threads all allocating/freeing 24/7 from a common shared heap - I designed and developed that as large Win32 API that used a single source code base to build a 32-bit and 64-bit versions of the API, all of that code (many thousands of lines) was system level programming, interacting with the Windows kernel object manager and so on, all of that was crafted to a good standard in C, I know the language well.

I mention this to underline the fact that I do understand the problem domain, I know C well, I've designed and built compilers and code generators in C, I'm not naïve as some of the comments here seem to want to insinuate.

So having said that allow me to respond:

When talking about low level programming languages –– which is what I understand 'a hardware "oriented" programming language' to mean ––, C is just the one with the best proven track record decades long.  It isn't that great, it's just the 'benchmark' one for others, due to its widespread use and role in systems programming and embedded development.

For examples of nearly bug-free programs written in C in the systems programming domain, go check out D. J. Bernstein's djbdns, qmail, daemontools, cdb.  This is the guy behind Curve25519, having released it in 2005.


Well as I say, I've written very sophisticated C heap APIs for concurrent threaded use 24/7 managing high speed incoming data feeds, in C. That code was and had to be able to run 24/7 it did, I had a set of principles and guidelines that helped to achieve that, some of these were specifically to plug "holes" that - like it or not - were attributable to the language itself, so yes one can write robust code in C but one can write robust code in assembler too! but that isn't an argument for assembler for C, the reality is one can write good code in C, but not because of C, but in spite of C.

Like it, dislike it, doesn't matter, C is just a tool.  But as a tool, its features and track record are significant.  So are its deficiencies, and that means any real effort to do better is valuable.

In comparison, C# is a managed language. .NET Micro requires at least 256k of RAM. .NET nanoFramework requires at least 64k of RAM, and runs on Cortex-M and RISC-V (ESP32-C3) cores.  So, perhaps suitable for medium to large embedded devices, but decidedly unsuitable for small ARMs and anything less than 32-bit architecures.

Ada can be used to program AVR 8-bit microcontrollers (see AVR-Ada), but it is still relatively little used.  One possible reason is that while GCC GNAT is GPL3+ licensed with a runtime library exception, AdaCore sells GNAT Pro, and the FSF/GCC GNAT is seen as "inferior", with the "proper" version being the sole product of a commercial company.  (Or maybe that's just me.)


C# is of course too large to run on resource constrained devices, the CLR is pretty sophisticated as is the garbage collector. It was also never aimed at MCUs, that's not a consideration the C# team have. I've used Nano framework and I've used TinyCLR too, these are fascinating and I was contributing to these once, the fact is there is no immediate prospect of these supporting generics or async/await, so that makes them very restrictive indeed as 99% of .Net library packages use generics and/or async, if MS would define a precise subset of .Net for MCUs then that's would help, but they haven't so the goals of these various initiatives is vague to put it mildly.

No, that's not it at all.  Not pointless, more like bass-ackwards.  We want the results too, we just have seen your approach before leading to nowhere.  We're trying to steer you to not repeat that, but actually produce something interesting.

If you start a language design from scratch, you must understand the amount of design choices already made for existing languages.  The ones in languages that have survived use in anger, are the ones where the choices support a programming paradigm the users find intuitive and effective.

Why did DiTBHo not start from scratch, and instead pared down C to a subset with some changes and additions, to arrive at their my-C, designed for strictly controlled and enforced embedded use cases?  Because they needed a tool fit for a purpose, and it was a straightforward way to achieve it.  Results matter.


Perhaps we need more clarity over the term "from scratch". I for one, have referred to PL/I several times for some of the strengths that language offered, that seems to me a good starting point, not starting from C I agree, but it's not starting from scratch by any means.

Okay, so if we believe that a ground-up design from scratch is unlikely to lead to an actual project solving the underlying problems OP (Sherlock Holmes) wants to solve, what would?


A grammar is necessary (not sufficient of course), unavoidable in fact so we need a grammar and I'm of the opinion that a grammar based on PL/I subset G is extremely attractive for reasons I've already articulated. A criticism I have of many, many newer languages like Rust, like Hare like Zig like Go like Swift even C# is that they are based on the C grammar and therefore do - unfortunately - stand to repeat some of the sins of the past.

Theory is nice, but practical reality always trumps theory.  Just because the ABNF of a language looks nice, doesn't mean it is an effective language.  As soon as you can compile running native binaries, start creating actual utilities – sort, grep, bc for example –, and look at the machine code the compiler generates.  Just because the code is nice and the abstractions just perfect, does not mean they are fit for generating machine code.  Compare the machine code to what the original language and other languages produce, when optimizations are disabled (for a more sensible comparison).


I suppose this might sound odd, but the generation of code, instruction sequence, object code that links and runs is - for me - rather academic, I've done it, it's obviously important but it isn't the most fruitful place to innovate in my opinion, code is code, instructions are instructions. Yes there is a lot involved but it's primarily about implementation, optimization not language, the language begins with the grammar, its expressivity.

One likely wouldn't see a huge glaring difference in the generated code from a good C compiler, a good Ada compiler and a good C++ compiler, at the end of the day they will look similar, loading registers, performing arithmetic, calculating addresses and so on.

Of course I accept that until one has produced linkable runnable code one doesn't really have a compiler, but I'm aware of that, I don't disagree at all.

What's frustrating in this thread that there are a few who just repeat the mantra "but you can do that in C if...", clearly those advocating that position are content with C and see no justification in even discussing a new language, in which case what's the point of participating in a thread about a new language!

I enjoyed your post, it is the kind of discourse that I hoped to see.











Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 05:52:13 pm
So, what specifically did you want to say about language grammar, extensibility, features that could be helpful to MCU developers?
There is very little (if anything) about language grammar that could be useful to MCU developers.

How did you establish that? what evidence supports this claim? Here's a post by an engineer (https://kevinlynagh.com/rust-zig/) that proves you wrong, I quote:

Quote
The first challenge I ran into with Rust was getting my firmware to run on hardware varying from 4-button dev-kit PCBs to the left/right halves of a wireless split to a single Atreus.
Varying the features of firmware at compile-time is known as “conditional compilation”. (It needs to be done at compile-time rather than run-time because microcontrollers have limited program space, roughly 10–100kB in my case.). Rust’s solution to this problem is “features”.

Quote
Conceptually Zig’s inline for is solving the same problem that Rust’s syntax macro solves (generating type-specific code at compile-time), but without the side quest of learning a lil’ pattern matching/expansion language. Rust has many language features and they’re all largely disjoint from each other, so knowing some doesn’t help me guess the others.

Quote
Conversely, this “consistency” principle also explains why I had such an easy time picking up Zig — it absolutely excels in this department. Not only are there many fewer features to learn in the first place, they seem to all fit together nicely: The comptime and inline for keywords, for example, allowed me to leverage at compile-time all the looping, conditions, arithmetic, and control flow I wanted using the syntax and semantics I’d already learned — Zig!

Anyway, moving on...


If extensibility is important, then domain specific libraries are the way to go.

They are certainly a way to go, I'm not advocating against libraries! A language that had no "volatile" specifier could be fine if it relied on a library to manipulate such data, but does that mean you therefore disapprove of C having a "volatile" keyword? Which do you think is preferable, a keyword for it or a library? Not to mention the obvious either but libraries also have to be written, they don't grow on trees!

Your initial questions were about far more than grammar, and therefore potentially more interesting. That's why people contributed.

You have consistently failed to understand that the main topic of importance is the runtime behaviour (in a broad sense) of something expressed in a language. Anything that improves the runtime behaviour is potentially important.

Please do not venture to speculate what you think I understand or do not understand, that's a disparaging remark, an ad-hominem.



Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 05:56:47 pm
So, what specifically did you want to say about language grammar, extensibility, features that could be helpful to MCU developers?
I'm saying that they are not at all important to microcontroller or embedded developers at all.

Sweeping speculation, if you have evidence I'd like to see it, otherwise it's opinion and we all have those. Also you too might find this of interest (https://kevinlynagh.com/rust-zig/).

I am saying that you are trying to solve a problem by daydreaming about irrelevant stuff, when you could be working on making things actually better.

Are you sure you're not a Windows C# enthusiast who dreams of creating a C#-like language that makes all embedded devices look like Windows or at least .Net runtime –– because "Windows is basically the entire world, after all", and being adulated for it?  You definitely sound like one.  I've met many, and all have either crashed or burned, or keep making unreliable unsecure shit and getting by with their social skills alone.

Very good, a strawman and ad-hominem all in one, if you can't argue logically, rationally,  factually then do not participate.

Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 06:22:25 pm
So, what specifically did you want to say about language grammar, extensibility, features that could be helpful to MCU developers?
There is very little (if anything) about language grammar that could be useful to MCU developers.

How did you establish that? what evidence supports this claim?

Observation over the decades. Basically MCU developers can swap grammars relatively easily and quickly. What trips them up is the behaviour, i.e. how the concepts expressed in a grammar (any grammar!) map onto the MCU's runtime behaviour.

Quote
Here's a post by an engineer (https://kevinlynagh.com/rust-zig/) that proves you wrong, I quote:

Quote
The first challenge I ran into with Rust was getting my firmware to run on hardware varying from 4-button dev-kit PCBs to the left/right halves of a wireless split to a single Atreus.
Varying the features of firmware at compile-time is known as “conditional compilation”. (It needs to be done at compile-time rather than run-time because microcontrollers have limited program space, roughly 10–100kB in my case.). Rust’s solution to this problem is “features”.

Quote
Conceptually Zig’s inline for is solving the same problem that Rust’s syntax macro solves (generating type-specific code at compile-time), but without the side quest of learning a lil’ pattern matching/expansion language. Rust has many language features and they’re all largely disjoint from each other, so knowing some doesn’t help me guess the others.

Quote
Conversely, this “consistency” principle also explains why I had such an easy time picking up Zig — it absolutely excels in this department. Not only are there many fewer features to learn in the first place, they seem to all fit together nicely: The comptime and inline for keywords, for example, allowed me to leverage at compile-time all the looping, conditions, arithmetic, and control flow I wanted using the syntax and semantics I’d already learned — Zig!

One data point (and a very arguable data point at that) does not constitute a convincing argument.

Quote

If extensibility is important, then domain specific libraries are the way to go.

They are certainly a way to go, I'm not advocating against libraries! A language that had no "volatile" specifier could be fine if it relied on a library to manipulate such data, but does that mean you therefore disapprove of C having a "volatile" keyword? Which do you think is preferable, a keyword for it or a librart?

The significance of a keyword is not that it is one symbol (of many symbols) in a grammar. The significance is how it is mapped onto runtime behaviour. The objective is to ensure that multiple sources that cause data mutation (threads, hardware registers, interrupts etc) have defined predictable useful behaviour. All that is necessary is that primitives (language and hardware) exist to express the necessary concepts.

There are, of course, several such low-level mechanisms described in the literature over the decades, and different languages include different low-level mechanisms. Those low-level mechanisms are usually "wrapped" into several more useful high-level conceptual mechanisms in the form of libraries expressing useful "Design Patterns", e.g. Posix threads or Doug Lea's Java Concurrency Library.

"Naked" use of the primitives (rather than well designed and debugged) libraries of design patterns is a frequent source of subtle unrepeatable errors.

Notice that the language grammar is completely irrelevant in that respect; the runtime behaviour is what's important.

Quote
Your initial questions were about far more than grammar, and therefore potentially more interesting. That's why people contributed.

You have consistently failed to understand that the main topic of importance is the runtime behaviour (in a broad sense) of something expressed in a language. Anything that improves the runtime behaviour is potentially important.

Please do not venture to speculate what you think I understand or do not understand, that's a disparaging remark, an ad-hominem.

I call 'em as I see 'em. Others are making similar observations.

I don't think you understand what an ad-hominem argument is and isn't.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 06:30:04 pm
So, what specifically did you want to say about language grammar, extensibility, features that could be helpful to MCU developers?
There is very little (if anything) about language grammar that could be useful to MCU developers.

How did you establish that? what evidence supports this claim?

Observation over the decades. Basically MCU developers can swap grammars relatively easily and quickly. What trips them up is the behaviour, i.e. how the concepts expressed in a grammar (any grammar!) map onto the MCU's runtime behaviour.

Quote
Here's a post by an engineer (https://kevinlynagh.com/rust-zig/) that proves you wrong, I quote:

Quote
The first challenge I ran into with Rust was getting my firmware to run on hardware varying from 4-button dev-kit PCBs to the left/right halves of a wireless split to a single Atreus.
Varying the features of firmware at compile-time is known as “conditional compilation”. (It needs to be done at compile-time rather than run-time because microcontrollers have limited program space, roughly 10–100kB in my case.). Rust’s solution to this problem is “features”.

Quote
Conceptually Zig’s inline for is solving the same problem that Rust’s syntax macro solves (generating type-specific code at compile-time), but without the side quest of learning a lil’ pattern matching/expansion language. Rust has many language features and they’re all largely disjoint from each other, so knowing some doesn’t help me guess the others.

Quote
Conversely, this “consistency” principle also explains why I had such an easy time picking up Zig — it absolutely excels in this department. Not only are there many fewer features to learn in the first place, they seem to all fit together nicely: The comptime and inline for keywords, for example, allowed me to leverage at compile-time all the looping, conditions, arithmetic, and control flow I wanted using the syntax and semantics I’d already learned — Zig!

One data point (and a very arguable data point at that) does not constitute a convincing argument.

I agree, but is so much better than no data points - if you get my drift. Your observations are fine, but all observations are interpretations of data, you cannot exclude your own biases and prejudices.

Quote

If extensibility is important, then domain specific libraries are the way to go.

They are certainly a way to go, I'm not advocating against libraries! A language that had no "volatile" specifier could be fine if it relied on a library to manipulate such data, but does that mean you therefore disapprove of C having a "volatile" keyword? Which do you think is preferable, a keyword for it or a librart?

The significance of a keyword is not that it is one symbol (of many symbols) in a grammar. The significance is how it is mapped onto runtime behaviour. The objective is to ensure that multiple sources that cause data mutation (threads, hardware registers, interrupts etc) have defined predictable useful behaviour. All that is necessary is that primitives (language and hardware) exist to express the necessary concepts.

There are, of course, several such low-level mechanisms described in the literature over the decades, and different languages include different low-level mechanisms. Those low-level mechanisms are usually "wrapped" into several more useful high-level conceptual mechanisms in the form of libraries expressing useful "Design Patterns", e.g. Posix threads or Doug Lea's Java Concurrency Library.

"Naked" use of the primitives (rather than well designed and debugged) libraries of design patterns is a frequent source of subtle unrepeatable errors.

Notice that the language grammar is completely irrelevant in that respect; the runtime behaviour is what's important.

Yes you keep saying that but you never answered the question - if libraries are the "way to go" do you think C should not have a "volatile" keyword and instead rely on a library? if you don't understand the question then say so, if you do then is your answer "yes" or "no" or "I don't know"?

Quote
Your initial questions were about far more than grammar, and therefore potentially more interesting. That's why people contributed.

You have consistently failed to understand that the main topic of importance is the runtime behaviour (in a broad sense) of something expressed in a language. Anything that improves the runtime behaviour is potentially important.

Please do not venture to speculate what you think I understand or do not understand, that's a disparaging remark, an ad-hominem.

I call 'em as I see 'em. Others are making similar observations.

I don't think you understand what an ad-hominem argument is and isn't.

A true classic oxymoron "I don't think you understand what an ad-hominem argument is" a bit like cheating on one's ethics final!

Let me say this, if you prefer debating to discussing the subject of the thread then feel free to start a thread or pick an existing thread in an appropriate thread or section of the forum and I'll give you some free lessons.

Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 07:15:47 pm
So, what specifically did you want to say about language grammar, extensibility, features that could be helpful to MCU developers?
There is very little (if anything) about language grammar that could be useful to MCU developers.

How did you establish that? what evidence supports this claim?

Observation over the decades. Basically MCU developers can swap grammars relatively easily and quickly. What trips them up is the behaviour, i.e. how the concepts expressed in a grammar (any grammar!) map onto the MCU's runtime behaviour.

Quote
Here's a post by an engineer (https://kevinlynagh.com/rust-zig/) that proves you wrong, I quote:

Quote
The first challenge I ran into with Rust was getting my firmware to run on hardware varying from 4-button dev-kit PCBs to the left/right halves of a wireless split to a single Atreus.
Varying the features of firmware at compile-time is known as “conditional compilation”. (It needs to be done at compile-time rather than run-time because microcontrollers have limited program space, roughly 10–100kB in my case.). Rust’s solution to this problem is “features”.

Quote
Conceptually Zig’s inline for is solving the same problem that Rust’s syntax macro solves (generating type-specific code at compile-time), but without the side quest of learning a lil’ pattern matching/expansion language. Rust has many language features and they’re all largely disjoint from each other, so knowing some doesn’t help me guess the others.

Quote
Conversely, this “consistency” principle also explains why I had such an easy time picking up Zig — it absolutely excels in this department. Not only are there many fewer features to learn in the first place, they seem to all fit together nicely: The comptime and inline for keywords, for example, allowed me to leverage at compile-time all the looping, conditions, arithmetic, and control flow I wanted using the syntax and semantics I’d already learned — Zig!

One data point (and a very arguable data point at that) does not constitute a convincing argument.

I agree, but is so much better than no data points - if you get my drift. Your observations are fine, but all observations are interpretations of data, you cannot exclude your own biases and prejudices.

Quote

If extensibility is important, then domain specific libraries are the way to go.

They are certainly a way to go, I'm not advocating against libraries! A language that had no "volatile" specifier could be fine if it relied on a library to manipulate such data, but does that mean you therefore disapprove of C having a "volatile" keyword? Which do you think is preferable, a keyword for it or a librart?

The significance of a keyword is not that it is one symbol (of many symbols) in a grammar. The significance is how it is mapped onto runtime behaviour. The objective is to ensure that multiple sources that cause data mutation (threads, hardware registers, interrupts etc) have defined predictable useful behaviour. All that is necessary is that primitives (language and hardware) exist to express the necessary concepts.

There are, of course, several such low-level mechanisms described in the literature over the decades, and different languages include different low-level mechanisms. Those low-level mechanisms are usually "wrapped" into several more useful high-level conceptual mechanisms in the form of libraries expressing useful "Design Patterns", e.g. Posix threads or Doug Lea's Java Concurrency Library.

"Naked" use of the primitives (rather than well designed and debugged) libraries of design patterns is a frequent source of subtle unrepeatable errors.

Notice that the language grammar is completely irrelevant in that respect; the runtime behaviour is what's important.

Yes you keep saying that but you never answered the question - if libraries are the "way to go" do you think C should not have a "volatile" keyword and instead rely on a library? if you don't understand the question then say so, if you do then is your answer "yes" or "no" or "I don't know"?

Sigh. That's a false dichotomy.

Not only isn't your question the right question, it isn't even the wrong question. It is, however, a reflection of the point I've been trying (and failing) to get you to understand: the difference between syntax/language and semantics/meaning/behaviour. Most people here care deeply about the latter, but don't care about the former.

The behaviour I mentioned above is needed, the keyword isn't. Whatever syntax and primitives are used, they will usually be wrapped up in a library.
Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 26, 2022, 07:22:28 pm
Even for the people who just want Carry, I highly doubt they are prepared for the different implementations of the carry flag with respect to subtraction and compare.

And it's simply not hard to express what you ACTUALLY WANT in portable C anyway. The compiler will automatically use the flags where available.

Bignums aren't easy in portable C :)

Well, yes they are, as just demonstrated.

Yep they are indeed.

I have implemented some kind of arbitrary precision library. It's able to do with various base integer widths without a problem. I've admittedly used some of the GCC's builtins to speed things up (such as the 'compute with overflow' kind of builtins), which themselves are reasonably portable if you stick to GCC, but I could have perfectly done without them and make the code 100% standard C code.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 07:54:13 pm
So, what specifically did you want to say about language grammar, extensibility, features that could be helpful to MCU developers?
There is very little (if anything) about language grammar that could be useful to MCU developers.

How did you establish that? what evidence supports this claim?

Observation over the decades. Basically MCU developers can swap grammars relatively easily and quickly. What trips them up is the behaviour, i.e. how the concepts expressed in a grammar (any grammar!) map onto the MCU's runtime behaviour.

Quote
Here's a post by an engineer (https://kevinlynagh.com/rust-zig/) that proves you wrong, I quote:

Quote
The first challenge I ran into with Rust was getting my firmware to run on hardware varying from 4-button dev-kit PCBs to the left/right halves of a wireless split to a single Atreus.
Varying the features of firmware at compile-time is known as “conditional compilation”. (It needs to be done at compile-time rather than run-time because microcontrollers have limited program space, roughly 10–100kB in my case.). Rust’s solution to this problem is “features”.

Quote
Conceptually Zig’s inline for is solving the same problem that Rust’s syntax macro solves (generating type-specific code at compile-time), but without the side quest of learning a lil’ pattern matching/expansion language. Rust has many language features and they’re all largely disjoint from each other, so knowing some doesn’t help me guess the others.

Quote
Conversely, this “consistency” principle also explains why I had such an easy time picking up Zig — it absolutely excels in this department. Not only are there many fewer features to learn in the first place, they seem to all fit together nicely: The comptime and inline for keywords, for example, allowed me to leverage at compile-time all the looping, conditions, arithmetic, and control flow I wanted using the syntax and semantics I’d already learned — Zig!

One data point (and a very arguable data point at that) does not constitute a convincing argument.

I agree, but is so much better than no data points - if you get my drift. Your observations are fine, but all observations are interpretations of data, you cannot exclude your own biases and prejudices.

Quote

If extensibility is important, then domain specific libraries are the way to go.

They are certainly a way to go, I'm not advocating against libraries! A language that had no "volatile" specifier could be fine if it relied on a library to manipulate such data, but does that mean you therefore disapprove of C having a "volatile" keyword? Which do you think is preferable, a keyword for it or a librart?

The significance of a keyword is not that it is one symbol (of many symbols) in a grammar. The significance is how it is mapped onto runtime behaviour. The objective is to ensure that multiple sources that cause data mutation (threads, hardware registers, interrupts etc) have defined predictable useful behaviour. All that is necessary is that primitives (language and hardware) exist to express the necessary concepts.

There are, of course, several such low-level mechanisms described in the literature over the decades, and different languages include different low-level mechanisms. Those low-level mechanisms are usually "wrapped" into several more useful high-level conceptual mechanisms in the form of libraries expressing useful "Design Patterns", e.g. Posix threads or Doug Lea's Java Concurrency Library.

"Naked" use of the primitives (rather than well designed and debugged) libraries of design patterns is a frequent source of subtle unrepeatable errors.

Notice that the language grammar is completely irrelevant in that respect; the runtime behaviour is what's important.

Yes you keep saying that but you never answered the question - if libraries are the "way to go" do you think C should not have a "volatile" keyword and instead rely on a library? if you don't understand the question then say so, if you do then is your answer "yes" or "no" or "I don't know"?

Sigh. That's a false dichotomy.

Not only isn't your question the right question, it isn't even the wrong question. It is, however, a reflection of the point I've been trying (and failing) to get you to understand: the difference between syntax/language and semantics/meaning/behaviour. Most people here care deeply about the latter, but don't care about the former.

The behaviour I mentioned above is needed, the keyword isn't. Whatever syntax and primitives are used, they will usually be wrapped up in a library.

Very well I'll tell you, the answer is in fact "no" because C with a volatile keyword is preferable to a library method invocation both syntactically and semantically and for performance reasons too.

You can argue otherwise if you want, I'm all ears, but dismissing the question because you'd rather not dignify it with an answer isn't logical, it's evasive.

Now you're at it again with "most people here care deeply about the latter, but don't care about the former" this is no doubt another of your personal observations and as I said all observations involve interpretation and all interpretation involves biases, prejudices and existing beliefs - upshot? it's a meaningless claim.

Now, what are examples of these semantic, behavioral concepts you "care deeply about" I'm as interested in this as I am grammar, but was focusing on grammar initially, if you want to start discussing behaviors then fine.

One cannot have a language without a grammar so one must - somehow - identify a grammar to use, it is an essential part of a programming language, you seem to be saying it is irrelevant, well if that were true use assembler or perhaps COBOL or RPG?

Finally as I just pointed out but you seem to have missed, libraries do not grow on trees, they are written and they are written using a language, you cannot wave away the language issue by replacing it with a library, all that does is move the code, it doesn't eliminate it.






Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 07:59:40 pm
Even for the people who just want Carry, I highly doubt they are prepared for the different implementations of the carry flag with respect to subtraction and compare.

And it's simply not hard to express what you ACTUALLY WANT in portable C anyway. The compiler will automatically use the flags where available.

Bignums aren't easy in portable C :)

Well, yes they are, as just demonstrated.

Yep they are indeed.

I have implemented some kind of arbitrary precision library. It's able to do with various base integer widths without a problem. I've admittedly used some of the GCC's builtins to speed things up (such as the 'compute with overflow' kind of builtins), which themselves are reasonably portable if you stick to GCC, but I could have perfectly done without them and make the code 100% standard C code.

When I last looked at the problem, c1986, it had to be portable across machines, the language was much simpler and any given compiler targeted, at best, a small subset of machines.

My comment was based on erroneously ignoring the changes.since then. Mea culpa.

I had implemented a 32 bit floating point package on a 6800 a decade earlier, in a medium level language. Without checking, I would have used the carry flag, which doesn't exist in C
Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 26, 2022, 08:18:40 pm
A grammar is necessary (not sufficient of course), unavoidable in fact so we need a grammar and I'm of the opinion that a grammar based on PL/I subset G is extremely attractive for reasons I've already articulated. A criticism I have of many, many newer languages like Rust, like Hare like Zig like Go like Swift even C# is that they are based on the C grammar and therefore do - unfortunately - stand to repeat some of the sins of the past.
I see grammar as an arbitrary choice, related more to what the designer considers "pretty" than anything else.
It is the language paradigm (generic approach to problem solving), and concepts, that matter.

If you start with a grammar, you are basically starting with the statement "This is the way all possible concepts in this programming language shall be expressed."

If you start at the conceptual level, you sketch out the features of the language, and can then choose a grammar that best suits the needs.

In my opinion, the only reason to start from the grammar upwards is if you 1) believe you already know everything you need to know about the language being designed, and/or 2) how it looks is of paramount importance to you.  (These are the two reasons I've seen in the wild, that is.)

Let's use my own gripes with C as a starting point, and examine how the above affects the design process and the result.

Because of embedded uses, I want to allow pointers to specific addresses and to individual objects, but I do not want them to be extensible to arrays, except via explicit constructs that also specify the size/range of the array.

I do not want to force the ABI to pass arrays as (origin, length) or (origin, step, length) tuples, because my aim is to be able to prove that all accesses are valid using static code analysis tools.  If a cosmic ray flips a bit and that causes a crash because of lack of runtime bounds checking, I'm okay with that.  (This, of course, is a design choice, and by no means the only one possible!)

I also like being able to redefine basic operators for custom object types.  I don't want to go whole-hog object-oriented, making the operator overloading object-specific; I'm fine with the operator overloading being purely based on static types only.  (I am aiming at small to medium-sized embedded devices, and the cost of the indirection via object pointers is too much for this niche, in my opinion.  Another design choice.)

I now cobble together something that seems to compile a small snippet of source code to pseudo-assembly.

Uh-oh, problem.  Because I do not force ABIs to explicitly pass a parameter describing the array length, I need a way to explicitly extract the origin and the length of an array.  I could use built-in functions or address and length operators.  Except that if I have already decided on a grammar, my choice is dictated by that grammar.  Oops.

In real life, both options have their upsides and downsides.  When the length is needed in a function where the array was passed as a parameter, we really do need to pass the length also.  There are several options for this particular design choice, but they all boil down to the fact that in a function specification, we need a way to specify that a parameter is the length of a specific array also passed as a parameter.  (With this information, my compiler can tell at compile-time whether the length operator or built-in function is allowed.)  This significantly affects the grammar, of course!
If my grammar was locked in already, I'd have to cobble together a less than optimal workaround.

Even though I have piles of experience myself, I know I am not aware of all the features and details beforehand.  But I also know that I can construct the grammar as I go along, and collect and implement all the features of the new programming language.  Having the grammar be defined by the features, instead of vice versa, gives me the most leeway.

For example, if I wanted this language to be mostly compatible with C otherwise, I would make the address-of and length-of operators, so that those porting code would need to pay attention to every case where a pointer is used as an array or vice versa.  C already has an address-of operator &, but it might be confusing, and make the static code analysis more difficult.  Instead, I might choose say @ and #, or origin and length (keyword operators, like sizeof).  But I do not know yet; I would experiment with it in practice –– on unsuspecting victims, preferably –– to see if they grasp the concept intuitively, and therefore would be likely to apply this to write memory-safer code.  Locking in the grammar up front makes such experiments irrelevant; the decision has been made already.

Thus, the obvious conclusion from going at it grammar-first is that it is grandiose, futile, waste of time, or all three.



Do not forget that others are not commenting about you personally, they are commenting your approach and what you describe in your output.

This is also why I use a pseudonym, instead of my real name: it helps me remember that any negative feedback I get is based on my output, my communications, and not my person.  I can change my communication style somewhat –– although verbosity seems to be a fixed feature for me ––, but there is no reason to think my person is under attack, so although discussion can get very heated at times, there is no reason to consider it as between persons: it is just a heated discussion.  In a different concurrent discussion with the same members, I can be (and have been), quite calm and relaxed –– again, because it is not about persons, the heat is only about the subject matter.  I think sometimes a little heat is good, because it shows that others care, and because it (at least for me) causes one to reconsider their own output, to see if the heat is on track, or just a mistargeted flamethrower.  Useful, in other words.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 08:20:58 pm
I had implemented a 32 bit floating point package on a 6800 a decade earlier, in a medium level language. Without checking, I would have used the carry flag, which doesn't exist in C

This caught my eye "carry flag" are you of the opinion this could be useful? if it had existed in C?
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 08:34:48 pm
I had implemented a 32 bit floating point package on a 6800 a decade earlier, in a medium level language. Without checking, I would have used the carry flag, which doesn't exist in C

This caught my eye "carry flag" are you of the opinion this could be useful? if it had existed in C?

It would have been convenient in that case, but different processors have very different semantics for their carry flag. Some calculate it for every instruction, but since it is on the critical path, others don't. For all I know there may be processors without a carry flag; I no longer bother to keep up with processor ISAs.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 09:01:54 pm

Yes you keep saying that but you never answered the question - if libraries are the "way to go" do you think C should not have a "volatile" keyword and instead rely on a library? if you don't understand the question then say so, if you do then is your answer "yes" or "no" or "I don't know"?

Sigh. That's a false dichotomy.

Not only isn't your question the right question, it isn't even the wrong question. It is, however, a reflection of the point I've been trying (and failing) to get you to understand: the difference between syntax/language and semantics/meaning/behaviour. Most people here care deeply about the latter, but don't care about the former.

The behaviour I mentioned above is needed, the keyword isn't. Whatever syntax and primitives are used, they will usually be wrapped up in a library.

Very well I'll tell you, the answer is in fact "no" because C with a volatile keyword is preferable to a library method invocation both syntactically and semantically and for performance reasons too.

Firstly a decent compiler will remove any inefficiency associated with a procedure call. (In Java's case it can even remove it at runtime based on how the code+data is actually using it; C# can only make compile time guesses)

Secondly the reason for wrapping a primitive in a procedure is to encapsulate and reuse well-thought out higher level concepts, rather that to re-create wheels that turn out not to be circular.

Examples of such design patterns are mailboxes, rendezvous, queues, counting semaphores, channels, timeouts, fork-join, etc etc. One example of many is that Doug Lea did a good job of pulling many of these together in his concurrency library; FFI see https://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/intro.html

Quote
Now, what are examples of these semantic, behavioral concepts you "care deeply about" I'm as interested in this as I am grammar, but was focusing on grammar initially, if you want to start discussing behaviors then fine.

I gave a quick list of them earlier in this thread; I'm not going to repeat them. It doesn't surprise me that you haven't noticed them.

Quote
One cannot have a language without a grammar so one must - somehow - identify a grammar to use, it is an essential part of a programming language, you seem to be saying it is irrelevant, well if that were true use assembler or perhaps COBOL or RPG?

I have done. Well not COBOL and RPG for real time applications, but multiple assemblers, macro languages, high level languages of various "persuasions", and even hardware design languages.

Quote
Finally as I just pointed out but you seem to have missed, libraries do not grow on trees, they are written and they are written using a language, you cannot wave away the language issue by replacing it with a library, all that does is move the code, it doesn't eliminate it.

Actually, in very deep and useful ways you can do exactly that.

An extreme and good case is a current commercial processor effectlvely implements "library level" concepts directly in hardware, e.g. channels, timeouts and even the RTOS.

Yes, the RTOS is in hardware. Yes, there are language constructs that make use of it. I've mentioned it earlier, but again I'm not surprised you haven't picked up on it.

Background: when I was in a research lab, the director stated that the only unforgiveable sin was to not know the literature. You need a wider understanding of what is out there, what works, and what the limitations might be. Without that you will, at best, take a long time inventing an elliptical wheel.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 09:03:04 pm
A grammar is necessary (not sufficient of course), unavoidable in fact so we need a grammar and I'm of the opinion that a grammar based on PL/I subset G is extremely attractive for reasons I've already articulated. A criticism I have of many, many newer languages like Rust, like Hare like Zig like Go like Swift even C# is that they are based on the C grammar and therefore do - unfortunately - stand to repeat some of the sins of the past.

I see grammar as an arbitrary choice, related more to what the designer considers "pretty" than anything else.
It is the language paradigm (generic approach to problem solving), and concepts, that matter.

I don't know what you mean by "pretty". Grammars have properties, not all grammars share the same identical properties, there are differences and those are things that should influence the choice of grammar. To glibly say the choice is arbitrary is tantamount to saying the properties do not matter, but that isn't true.

If you start with a grammar, you are basically starting with the statement "This is the way all possible concepts in this programming language shall be expressed."

If you start at the conceptual level, you sketch out the features of the language, and can then choose a grammar that best suits the needs.

Very well, so what at the conceptual level do you think should play into the grammar?

In my opinion, the only reason to start from the grammar upwards is if you 1) believe you already know everything you need to know about the language being designed, and/or 2) how it looks is of paramount importance to you.  (These are the two reasons I've seen in the wild, that is.)

I don't think its very easy to devise a programming language grammar that is 100% free from the influence of other previous languages. There's no such thing as a totally, brand new, fresh grammar, or it's very very rare. APL could be considered an example of that though, I suppose.

Consider too C#, Java, C++, Swift, Objective-C, Perl, PHP, JavaScript, Go, Rust and more, all are regarded as being derived from the C grammar. So all of these must have selected - up front - C as the basis for their grammar.

As for PL/I there are aspects of its grammar that I think are valuable and that's why it is my starting point. Yes the problem domain is important but few have wanted to discuss that here, I've asked what kind of things an MCU developer might find helpful but sadly there's been rather a lot of disparaging remarks and not that much about actual requirements.

Let's use my own gripes with C as a starting point, and examine how the above affects the design process and the result.

Because of embedded uses, I want to allow pointers to specific addresses and to individual objects, but I do not want them to be extensible to arrays, except via explicit constructs that also specify the size/range of the array.

I do not want to force the ABI to pass arrays as (origin, length) or (origin, step, length) tuples, because my aim is to be able to prove that all accesses are valid using static code analysis tools.  If a cosmic ray flips a bit and that causes a crash because of lack of runtime bounds checking, I'm okay with that.  (This, of course, is a design choice, and by no means the only one possible!)

I also like being able to redefine basic operators for custom object types.  I don't want to go whole-hog object-oriented, making the operator overloading object-specific; I'm fine with the operator overloading being purely based on static types only.  (I am aiming at small to medium-sized embedded devices, and the cost of the indirection via object pointers is too much for this niche, in my opinion.  Another design choice.)

I now cobble together something that seems to compile a small snippet of source code to pseudo-assembly.

Uh-oh, problem.  Because I do not force ABIs to explicitly pass a parameter describing the array length, I need a way to explicitly extract the origin and the length of an array.  I could use built-in functions or address and length operators.  Except that if I have already decided on a grammar, my choice is dictated by that grammar.  Oops.

In real life, both options have their upsides and downsides.  When the length is needed in a function where the array was passed as a parameter, we really do need to pass the length also.  There are several options for this particular design choice, but they all boil down to the fact that in a function specification, we need a way to specify that a parameter is the length of a specific array also passed as a parameter.  (With this information, my compiler can tell at compile-time whether the length operator or built-in function is allowed.)  This significantly affects the grammar, of course!
If my grammar was locked in already, I'd have to cobble together a less than optimal workaround.

Those are not strictly grammar questions (well operator overloading is). How the sizes of arrays is represented and accessible at runtime is a semantic issue. One could add that to C, one could write a C compiler that looks like C but implements arrays differently to normal, in principle anyway. Arrays can be described by simple metadata, in fact in PL/I (as well as Pascal and Fortran and COBOL...) there 's no need to care because that metadata is always present under the hood. It's a tiny amount of data too.

IMHO this is something I'd absolutely include in a new language, I'd also support optional user specified bounds too, that way we could for example, declare a 2D arrays of doubles where there subscripts range from (say) -9 thru +9 and support that for an arbitrary number of dimensions.

PL/I also supported the novel "isub" defined arrays. In that abstraction we can declare a 2D 10 x 10 array and then declare another 1D 10 element array where the elements of the latter map to the elements of the former in some defined way. So one could access the diagonal of a matrix just by a suitable declaration no logic, no code, no bugs.

This is achieved using the "defined" storage class basically meaning overlayed, like a "union" but much less restricted. A similar thing is true for strings, these can be fixed length or varying length and what's more we can declare a parameter to have a "*" length, that means you can pass it any fixed length string and the called code can easily find the length - even fixed length strings have length metadata.

The engineers that created PL/I were very experienced assembler programmers too, they were expert IBM mainframe programmers who understood hardware very well indeed. Many of the things included in the language stem from that, overlaying different variables on a common address, accessing data through implicit pointers, not just explicit pointers as in C.

Even though I have piles of experience myself, I know I am not aware of all the features and details beforehand.  But I also know that I can construct the grammar as I go along, and collect and implement all the features of the new programming language.  Having the grammar be defined by the features, instead of vice versa, gives me the most leeway.

For example, if I wanted this language to be mostly compatible with C otherwise, I would make the address-of and length-of operators, so that those porting code would need to pay attention to every case where a pointer is used as an array or vice versa.  C already has an address-of operator &, but it might be confusing, and make the static code analysis more difficult.  Instead, I might choose say @ and #, or origin and length (keyword operators, like sizeof).  But I do not know yet; I would experiment with it in practice –– on unsuspecting victims, preferably –– to see if they grasp the concept intuitively, and therefore would be likely to apply this to write memory-safer code.  Locking in the grammar up front makes such experiments irrelevant; the decision has been made already.

Thus, the obvious conclusion from going at it grammar-first is that it is grandiose, futile, waste of time, or all three.

Do not forget that others are not commenting about you personally, they are commenting your approach and what you describe in your output.

Well we can't not define a grammar AND develop a new language at the same time. I've asked what kind of things would be helpful in a new language because I want  to set boundaries for a grammar, I've based it on PL/I because that has some desirable properties not seen in other grammars, like no reserved words, so this is not specifically a PL/I grammar it is a grammar that - like PL/I - will have no reserved words but unlike PL/I would use { and } for block delimiters. At the time PL/I was active in industry people did not have keyboards, very few people used a keyboard or a terminal, they wrote code - I wrote code - on coding sheets and these were then "punched in" by the "punch girls". I'd get compiler listing the next day, syntax errors and all.

The point I'm making is that some things in the original IBM PL/I grammar like being all UPPERCASE and terms like DO and DO WHILE and IF X THE DO... etc are primarily there because of the nature of the industry at that time. Writing a { or a } on a coding sheet was a risk, it could easily by misread. But not today, so things like that can be discarded and we can use "{" and "}" rather than "do;" and "end;"

Someone else mentioned that "the carry flag isn't available in C" well this is the kind if thing I'd like to hear more about, would it, could it, be helpful to expose such a concept in the language?

This is also why I use a pseudonym, instead of my real name: it helps me remember that any negative feedback I get is based on my output, my communications, and not my person.  I can change my communication style somewhat –– although verbosity seems to be a fixed feature for me ––, but there is no reason to think my person is under attack, so although discussion can get very heated at times, there is no reason to consider it as between persons: it is just a heated discussion.  In a different concurrent discussion with the same members, I can be (and have been), quite calm and relaxed –– again, because it is not about persons, the heat is only about the subject matter.  I think sometimes a little heat is good, because it shows that others care, and because it (at least for me) causes one to reconsider their own output, to see if the heat is on track, or just a mistargeted flamethrower.  Useful, in other words.

I have a simple rule - never write anything to or about a person in a forum that I would not be totally willing and comfortable to say to them in person, in a meeting with others present, other team members, visitors to the company, vendors even HR, I try to communicate with that rule in mind, I sometimes fail but I really do try, many here do not adhere to such a rule, they speak and accuse and insult in a way that would be very unlikely to do in a group meeting without making fools of themselves or getting reprimanded by someone more senior.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 09:32:13 pm
I don't think its very easy to devise a programming language grammar that is 100% free from the influence of other previous languages. There's no such thing as a totally, brand new, fresh grammar, or it's very very rare. APL could be considered an example of that though, I suppose.

I've used LISP, Prolog, Forth, Smalltalk, Algol-60, various hardware definition languages. They all have radically different syntaxes.

Quote
Consider too C#, Java, C++, Swift, Objective-C, Perl, PHP, JavaScript, Go, Rust and more, all are regarded as being derived from the C grammar. So all of these must have selected - up front - C as the basis for their grammar.

And there you unwittingly demonstrate our point. The interesting and important differences between those languages has nothing to do with grammar.

Quote
Someone else mentioned that "the carry flag isn't available in C" well this is the kind if thing I'd like to hear more about, would it, could it, be helpful to expose such a concept in the language?

There are far more important concepts to expose in a language. Concentrate on those.

Start by understanding the concepts in https://www.jameswhanlon.com/the-xc-programming-language.html (https://www.jameswhanlon.com/the-xc-programming-language.html) amd https://www.xmos.ai/download/XMOS-Programming-Guide-(documentation)(F).pdf (https://www.xmos.ai/download/XMOS-Programming-Guide-(documentation)(F).pdf)
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 09:40:42 pm
I don't think its very easy to devise a programming language grammar that is 100% free from the influence of other previous languages. There's no such thing as a totally, brand new, fresh grammar, or it's very very rare. APL could be considered an example of that though, I suppose.

I've used LISP, Prolog, Forth, Smalltalk, Algol-60, various hardware definition languages. They all have radically different syntaxes.

Quote
Consider too C#, Java, C++, Swift, Objective-C, Perl, PHP, JavaScript, Go, Rust and more, all are regarded as being derived from the C grammar. So all of these must have selected - up front - C as the basis for their grammar.

And there you unwittingly demonstrate our point. The interesting and important differences between those languages has nothing to do with grammar.

Who said otherwise? The fact is that some of the problems common to all of them is attributable to the grammar - so in fact you unwittingly conformed my point!

Quote
Someone else mentioned that "the carry flag isn't available in C" well this is the kind if thing I'd like to hear more about, would it, could it, be helpful to expose such a concept in the language?

There are far more important concepts to expose in a language. Concentrate on those.

Start by understanding the concepts in https://www.jameswhanlon.com/the-xc-programming-language.html (https://www.jameswhanlon.com/the-xc-programming-language.html) amd https://www.xmos.ai/download/XMOS-Programming-Guide-(documentation)(F).pdf (https://www.xmos.ai/download/XMOS-Programming-Guide-(documentation)(F).pdf)

I will take a peek, how many people here use XC to write software on MCUs? It seems to bear some similarities to Occam. Are you trying to say that some of things in XC would be useful in a new language? If there are specific features why not call them out? I've asked several times and seen a lackluster response, even here you are just glibly telling me to read some manual rather than being clear about the features you value.

You're speaking about parallelism? message passing? what exactly? concentrate on my questions.
 



Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 26, 2022, 10:15:31 pm
I note that one of the desires there was "give easy access to the flags". I pointed out that many ISAs don't even *have* flags.
Yep.  In particular, you helped me realize I did not actually desire access to flags, just multiple function result values, to fulfill my needs.

Yes. Multiple return values are very desirable. Returning structures can substitute, but it's notationally annoying.

Most modern ABIs allow returning two word-size or one double sized value. With register based ABIs, usually in the same registers as used for the first two arguments. x86_64 is a weird exception with the first argument passed in RDI but the result in RAX.

But why can't you pass the same number of results as arguments? Function return is semantically no different to function call -- it is "calling the continuation". It would clearly be very very easy to enable this in modern ABIs that pass up to 6 or 8 arguments in registers -- it's simply relaxing a restriction. Figuring out where to put return values that overflow into RAM is harder. If the caller knows how many results are to be returned then it can allocate stack space for max(arguments, results). Support an unbounded number of return values would be a lot harder. But it's also hard to see a reason to support it. VARARGS seems like mostly a hack to support printf and friends.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 10:31:22 pm
I used a naked "8-pin Arduino" in my Vetinari clock (https://entertaininghacks.wordpress.com/2015/02/23/vetinari-digital-clock/), because that was the quick way to get an undemanding result.

But when making something that required very low power consumption, I simply wrote C to peek/poke the atmega328 registers. That avoided me having to work out how Arduino libraries might be frustrating me.
That's exactly why ATmega32u4 is one of my favourite 8-bitters.  It has native full-speed USB (12 Mbit/s, can do 1 Mbyte/sec even over USB Serial), but still is completely programmable without any HAL or similar, on bare metal, using nothing but avr-gcc (and avr-libc, if you really want), in (freestanding) C or C++.

I note that one of the desires there was "give easy access to the flags". I pointed out that many ISAs don't even *have* flags.
Yep.  In particular, you helped me realize I did not actually desire access to flags, just multiple function result values, to fulfill my needs.

An extremely common pattern for functions is to return a value or error.  Obviously, with scalar return types, when you return a single scalar, some values must be reserved for error codes.  (That can get messy, like it once did in the Linux kernel, where certain filesystem drivers could not handle reads over 231 bytes (due to widespread use of int), and returned negative values.  Now, the Linux kernel limits all single read() and write() syscalls to 231 less one page automagically.)  Having a separate value and status returns is sometimes more robust, even if the status return is just a single "OK"/"Error" flag.

However, SysV ABI on x86-64 (https://wiki.osdev.org/System_V_ABI#x86-64) (LP64, so ints are 32-bit, long and pointers are 64-bit) as used on Linux, macOS, FreeBSD, and others, already supports this (by using rax and rdx registers for 128-bit non-FP return values), so code like
    typedef struct { long x, y } ipair;
    lpair somefunc(long x, long y) { return (lpair){ .x = x, .y = y }; }
compiles to
    somefunc:
        mov  rax, rdi
        mov  rdx, rsi
        ret
because rdi and rsi are the first two (non-FP) parameters, and the pair of return values are returned in rax and rdx.
This suffices for my needs, although it is slightly odd-looking at first, wrapping return values into structs.  It would be nicer to have syntactic sugar for this and not require struct use, but this works too.

Because this is architecture ABI, any compiler for this architecture can use this.  Other ABIs, however, need to do annoying "emulation" by e.g. passing a pointer to the additional return values, similar to how larger (struct-type) result values are passed.  A compiler can obviously use its own function call ABI internally, but it does have to have some kind of foreign function interface* for bindings to binary libraries (if used on hosted environments/full OSes, and not just bare metal embedded).  It might be useful to use a rarely used register for the status/error return, so that it is not so often clobbered by ordinary code.

Simply put, I wasn't hankering so much for syntactic sugar, but for more sensible ABIs like SysV on x86-64.  Brucehoult put me straight on this, albeit indirectly.  :-+



* Part of the reason I like using Python for graphical user interfaces is that it has a very nice interface to calling native code, without needing any native code support; using pure Python code for the interface.  Certain libraries, like glib's gobject introspection for C, provide introspection information (.gi or gir files) that means full interfaces can be constructed without any bespoke code (machine or Python), using the introspection information only.  It also means that when the library and corresponding introspection file is updated, the new features are immediately available in Python also.  Very useful.

Because native machine code is dynamically loaded for access from Python, this also provides a perfect mechanism for license separation.  You can keep the user interface open source, modifiable by end users, but keep all Sekret Sauce proprietary code in your binary-only dynamic library.  In my experience, this provides the optimal balance between user access and proprietary requirements when going full open source isn't viable, e.g. for commercial reasons.

The C# language has options "ref" and "out". With these we can designate a parameter as "out" meaning it must, will, be assigned a value within the callee that will be visible to the caller.

One of the patterns we see often in C# code is the "try" pattern:

Code: [Select]

if (TryGetRemoteSystemStatus(system_id, var out status))
   {
   // do stuff.
   }


The language only lets the "status" be accessible inside the { } block of the if statement. It's scope (when used as shown) is restricted to that block.

In the function TryGetRemoteSystemStatus we return a bool, a false if we can't get what we want. Assuming the function has been written sensibly, it's then close to impossible for calling code to access the "status" when it should not. This a good pattern because it avoids the "if (status != null)..." pattern which relies in "null" as a sentinel.


So with "out" a write must take place to the variable within the callee before returning to the caller and with "ref" a write must take place to the variable within the caller before calling the callee.
Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 26, 2022, 10:35:49 pm
You can pass and return structures in C, which makes your life much easier. Yes the downside is that you need 1/ to define the structure first and 2/ to access its members. Sure a bit more syntax overhead. Interesting to note that very few developers ever use that feature.

I have mixed "feelings" about returning multiple values though. A few reasons for this, but one of them being the "implicit" order. Note that this is the exact same issue with function parameters. Ada fixes it by using parameter designators, although those are optional. But at least function parameters have an identifier. Many languages handle returning multiple values using some kind of tuples - so no identifier per se associated with each value returned inside the tuple, only some guaranteed order.

Most of that is mainly syntax sugar coating anyway.

And, tuples could be used in other contexts - not necessarily just return values of functions. To make it consistent, the list of function parameters would itself be a tuple. And tuples could be used as expressions.

The usual cute example is: 'x, y = y, x ' for swapping x and y.
Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 26, 2022, 10:39:21 pm
So, what specifically did you want to say about language grammar, extensibility, features that could be helpful to MCU developers?
There is very little (if anything) about language grammar that could be useful to MCU developers.
How did you establish that? what evidence supports this claim? Here's a post by an engineer (https://kevinlynagh.com/rust-zig/) that proves you wrong, I quote:
Ehh, what? :o

What has that post by that engineer have to do with language grammar?  They're talking about conditional builds, which is solved by any toolchain that generates linkable object files by selecting which object files to compile and include in the final image.

ELF object files support per-symbol sections, regardless of the language you use, which allows the linker to select which symbols (usually functions) to include based on which symbols may be referenced.  It means that only features you actually use are included in the linked target, and this is heavily used in current embedded toolchains, which basically all use ELF object files.  You do not even have to deselect the features you do not need; all you need to do is make sure all you might need is available (via whatever include or import mechanism your language uses), and use the ones you want.  Only the features you use, and the features they refer to, will get included in the final linked object.
_ _ _

The way we established "language grammar isn't important to embedded developers" is by being, and working with embedded developers, who use whatever tools fit the task at hand best.  Grammar is just flavour.  I've never heard of any successful developer choosing their programming language or environment based on the grammar.  I have seen many who did choose their programming language based on grammar or because it was the only language they could use, fail.  I've talked with, and continuously talk to, and occasionally help, many developers, both in embedded and application development.  Grammar has never been more than a flavour on top, as in "I find it ugly" or "it looks neat"; never at the core of any discussion related to solving a problem.

I'm not sure if I have talked shop in such details with the commonly suggested 1000 persons sufficient for a random opinion poll, and the selection definitely isn't random, but it is a significant number, well over a hundred at minimum.  I also do not shy away from antagonistic discussions, so I often disagree with others –– often to find out the underlying reasons for their opinions, because opinions themselves aren't useful, but the reasons underlying the opinions are.  (For one, opinions cannot be compared in any rational manner, but the reasons for those opinions can usually be rationally examined and compared.)

I note that one of the desires there was "give easy access to the flags". I pointed out that many ISAs don't even *have* flags.
Yep.  In particular, you helped me realize I did not actually desire access to flags, just multiple function result values, to fulfill my needs.
Yes. Multiple return values are very desirable. Returning structures can substitute, but it's notationally annoying.

Most modern ABIs allow returning two word-size or one double sized value. With register based ABIs, usually in the same registers as used for the first two arguments. x86_64 is a weird exception with the first argument passed in RDI but the result in RAX.

But why can't you pass the same number of results as arguments? Function return is semantically no different to function call -- it is "calling the continuation". It would clearly be very very easy to enable this in modern ABIs that pass up to 6 or 8 arguments in registers -- it's simply relaxing a restriction. Figuring out where to put return values that overflow into RAM is harder. If the caller knows how many results are to be returned then it can allocate stack space for max(arguments, results). Support an unbounded number of return values would be a lot harder. But it's also hard to see a reason to support it. VARARGS seems like mostly a hack to support printf and friends.
Exactly.  No, I don't need unbounded parameters or return values myself; even for printf and friends I prefer a string-building approach (if you recall my old replacement for standard C thread).

If we consider things like Foreign Function Interfaces and such, we could augment object file symbol versioning (or similar) to explicitly specify the exact function ABI variant, as in how many inputs and outputs, in which registers or stack –– noting that many architectures like x86-64 and ARMv8 already have separate vector registers for FP parameters.  It does not even need to know the logical types, ie. whether something is an address or a number, just which registers or stack words are used for inputs, outputs, and as scratch registers.  One would still need the exact declarations in each programming language, and compilers would need to check the declaration matches the object file ABI variant.

This is not dependent on any specific programming language, though.

It would be very interesting to consider if binary-only libraries should contain introspection information for their symbols, including the above information.  Not necessarily introspection in the same file, but in an associated file, in some kind of common format.  We already know from the widely used GObject Introspection (https://gi.readthedocs.io/en/latest/) (for FFI bindings of GObject-based libraries, to a number of programming languages) that it can be extremely useful and powerful mechanism for run-time linkage across programming languages.  (Most users of it aren't even aware of using it, it works that well.)
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 26, 2022, 10:49:15 pm
So, what specifically did you want to say about language grammar, extensibility, features that could be helpful to MCU developers?
There is very little (if anything) about language grammar that could be useful to MCU developers.
How did you establish that? what evidence supports this claim? Here's a post by an engineer (https://kevinlynagh.com/rust-zig/) that proves you wrong, I quote:
Ehh, what? :o

What has that post by that engineer have to do with language grammar?  They're talking about conditional builds, which is solved by any toolchain that generates linkable object files by selecting which object files to compile and include in the final image.

Yes, one language grammar lets him express it much better than the other.

ELF object files support per-symbol sections, regardless of the language you use, which allows the linker to select which symbols (usually functions) to include based on which symbols may be referenced.  It means that only features you actually use are included in the linked target, and this is heavily used in current embedded toolchains, which basically all use ELF object files.  You do not even have to deselect the features you do not need; all you need to do is make sure all you might need is available (via whatever include or import mechanism your language uses), and use the ones you want.  Only the features you use, and the features they refer to, will get included in the final linked object.

Yes you can jump through hoops or you can simplify things by empowering the programming language.

The way we established "language grammar isn't important to embedded developers" is by being, and working with embedded developers, who use whatever tools fit the task at hand best.  Grammar is just flavour.  I've never heard of any successful developer choosing their programming language or environment based on the grammar. 

Your conflating two things reasons for designing a language grammar and reasons for choosing a language, I never ever said these were the same. Your reasons for - say - choosing a gun will likely not be the same as the reasons the manufacturer has for designing that gun the way they did.

I have seen many who did choose their programming language based on grammar or because it was the only language they could use, fail.  I've talked with, and continuously talk to, and occasionally help, many developers, both in embedded and application development.  Grammar has never been more than a flavour on top, as in "I find it ugly" or "it looks neat"; never at the core of any discussion related to solving a problem.

And? what does that prove? your perception is your perception. And what does "ugly" mean, first you used "pretty" and now "ugly" what - do you mean by these terms? What is undesirable about "ugly" code and what is appealing about "pretty" code? Is one preferable to the other? if so, in what way? why?

I'm not sure if I have talked shop in such details with the commonly suggested 1000 persons sufficient for a random opinion poll, and the selection definitely isn't random, but it is a significant number, well over a hundred at minimum.  I also do not shy away from antagonistic discussions, so I often disagree with others –– often to find out the underlying reasons for their opinions, because opinions themselves aren't useful, but the reasons underlying the opinions are.  (For one, opinions cannot be compared in any rational manner, but the reasons for those opinions can usually be rationally examined and compared.)

So please don't present personal anecdotal perceptions as being anything more than that, this is why you'll sometimes see "IMHO" inside some of my posts.



Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 26, 2022, 11:09:15 pm
You can pass and return structures in C, which makes your life much easier. Yes the downside is that you need 1/ to define the structure first and 2/ to access its members. Sure a bit more syntax overhead. Interesting to note that very few developers ever use that feature.
Things get very interesting when you start using e.g. in C
    typedef {
        type   *data;
        size_t  size;
    } region;
because then you can use
    region  allocate(size_t minimum, size_t maximum);
    region  reallocate(region, size_t minimum, size_t maximum);

If you use a triplet,
    typedef {
        type   *data;
        size_t  size;
        size_t  used;
    } buffer;
then your allocation and reallocation routines can examine the amount of data (size), and only keep that, discarding the rest, even when increasing the size of the allocation.  Letting the library choose the exact size within a range, means the library can use much more time-efficient allocators without wasting memory, because when they have a larger slot/slab/slub to fulfill the allocation, they can examine the maximum, and provide either the entire region, or a suitably size-aligned one.  This is an extremely common pattern when reading data from a socket or a pipe (e.g. network comms), and the current message does not fit in the existing buffer, and the exact message length is not known beforehand (e.g. most HTTP transfers, WebSockets, etc.).

I have mixed "feelings" about returning multiple values though. A few reasons for this, but one of them being the "implicit" order. Note that this is the exact same issue with function parameters. Ada fixes it by using parameter designators, although those are optional. But at least function parameters have an identifier. Many languages handle returning multiple values using some kind of tuples - so no identifier per se associated with each value returned inside the tuple, only some guaranteed order.
I do not know what kind of syntax would work best, either.

Currently, my opinion is shaded by my ubiquitous use of Python tuples in some of my 2D/3D code.  Basically, I subclass tuples for quite efficient 2D/3D vectors, with vector-scalar and vector-vector operators implemented by the subclass.  I also use four-tuples for versors/bivectors, with just intra-class operators, and an accessor that yields a 3D rotation matrix, or a 4×4 transformation matrix (using the convention of (x,y,z,1) referring to 3D position vectors, and (x,y,z,0) referring to 3D direction vectors) with a caller-specified translation pre- or post-rotation.

I do use structs –– actually, structs containing unions with structs and vectorized types, because that happens to be compatible on all architectures I have that have SIMD –– in a similar fashion in C.  On those architectures, they're passed and returned in the vector registers, even though the C API looks normal, i.e.
    vec3d  vec3d_cross(const vec3d, const vec3d);
    double  vec3d_dot(const vec3d, const vec3d);
    vec3d  rot3d_apply(const rot3d, const vec3d);
    vec3d  trans3d_position(const trans3d, const vec3d);
    vec3d  trans3d_direction(const trans3d, const vec3d);
    float  vec2f_dot(const vec2f, const vec2f);
    float  vec2f_cross(const vec2f, const vec2f);
    vec2f  rot2f_apply(const rot2f, const vec2f);
    vec2f  trans2f_position(const trans2f, const vec2f);
    vec2f  trans2f_direction(const trans2f, const vec2f);
and so on, showing typical operations on 3D double-precision floating point vectors, 2D single-precision floating point vectors, 3D 3×3 rotation and 4×4 (3×4) transformation matrices, and 2D 2×2 rotation and 3×3 (2×3) transformation matrices.  Where _position applies the translation, _direction does not.

So, I am definitely biased towards something like (a, b) syntax describing a tuple.  Don't care what the exact grammar would be, though; I'll adapt.
Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on November 26, 2022, 11:11:01 pm
So please don't present personal anecdotal perceptions as being anything more than that, this is why you'll sometimes see "IMHO" inside some of my posts.

Ah, I see.  When you post someones personal anecdotal perceptions, they are 'proof'.
When I describe my own experiences discussing these technical details with well over a hundred other embedded developers, they're just 'perceptions', and not indicative of anything.

You, sir, are full of shit.

I'm out.  I'm not ready to try to help another electrodacus see the reality from their own preconceptions, sorry.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 11:33:51 pm
concentrate on my questions.
.

We are not here at your beck and call, nor to spoon feed you.

You have been pointed towards some answers to your questions. Now you need to read (not skim) the points, and do the work to understand the literature.
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 26, 2022, 11:36:55 pm
When I last looked at the problem, c1986, it had to be portable across machines, the language was much simpler and any given compiler targeted, at best, a small subset of machines.

My comment was based on erroneously ignoring the changes.since then. Mea culpa.

I had implemented a 32 bit floating point package on a 6800 a decade earlier, in a medium level language. Without checking, I would have used the carry flag, which doesn't exist in C

Comparing the language called "C" and compilers for it in the 1970s or 1980s to the language called "C" in 2022 is pretty much pointless.

Early compilers produced a very literal translation that gave code often many times slower than hand written assembly language. Most compilers could target only one ISA. If they were "portable" then they made even worse code. If they were free then they were probably even worse again -- and even paying $10k+ got you something pretty awful.

Modern gcc and llvm have very sophisticated machine-independent optimisation pipelines that can recognise patterns in portable C code that map to low level machine instructions -- for example to map unsigned "(a+b) < a" to using the carry flag on ISAs that have one. As demonstrated a few messages up. And to use SETcc instructions on machines that have those. And MOVcc instructions on machines that have those. And use an actual conditional branch if/then/else as a last resort if the machine doesn't have any of the above.

You won't get that from a 1970s compiler.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 26, 2022, 11:38:17 pm
So please don't present personal anecdotal perceptions as being anything more than that, this is why you'll sometimes see "IMHO" inside some of my posts.

Ah, I see.  When you post someones personal anecdotal perceptions, they are 'proof'.
When I describe my own experiences discussing these technical details with well over a hundred other embedded developers, they're just 'perceptions', and not indicative of anything.

You, sir, are full of shit.

I'm out.  I'm not ready to try to help another electrodacus see the reality from their own preconceptions, sorry.

FWIW, those were my thoughts too.

Control-d
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 26, 2022, 11:54:53 pm
It would have been convenient in that case, but different processors have very different semantics for their carry flag.

Exactly correct.

Quote
For all I know there may be processors without a carry flag; I no longer bother to keep up with processor ISAs.

MIPS
DEC Alpha
Intel IA64
RISC-V

Probably others I can't think of right now.
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 27, 2022, 12:15:47 am
Your position is pessimistic, discouraging, not open minded, this is nothing to do with your undoubted credentials either, it is you, your hostile tone. This is just a discussion and you're frothing and fuming.

The only frothing is coming at me, not from me.

I see others I respect have now reached and expressed the same conclusion as I did. To their credit, they perhaps gave the benefit of the doubt for longer, though not forever.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 27, 2022, 01:22:10 am
For all I know there may be processors without a carry flag; I no longer bother to keep up with processor ISAs.

MIPS
DEC Alpha
Intel IA64
RISC-V

Probably others I can't think of right now.

I thought I remembered MIPS not having them, and I presume that would carry over [sic] into RISC-V.

If Alpha had crossed my mind, I would have presumed it didn't, since that went to all extremes to excise everything they could. (And, IIRC, excised byte addressing - oops).

I'm not sure what the other currently interesting architecture (the Mill) does. That plus CHERI do raise hopes for the future, and are worth watching. Interestingly they both start from the position that even though it is a right pain in the backside, pragmatically it is necessary to be able to run C efficiently.
Title: Re: A new, hardware "oriented" programming language
Post by: DiTBho on November 27, 2022, 01:54:00 am
MIPS
DEC Alpha
Intel IA64
RISC-V

Probably others I can't think of right now.

also, subtract with borrow (68k) vs subtract with carry (ppc)
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 27, 2022, 01:59:26 am
If Alpha had crossed my mind, I would have presumed it didn't, since that went to all extremes to excise everything they could. (And, IIRC, excised byte addressing - oops).

Alpha always had byte addressing. What it doesn't have, for the first half dozen models (until 199's "EV56" 21164A), was byte (and 16 bit) load and store instructions. It wasn't a raw speed thing -- Alpha was often the fastest CPU in the world during that time, including on software doing a lot of byte operations.

The problem was multiprocessor systems and software that assumes byte accesses are atomic.

Quote
I'm not sure what the other currently interesting architecture (the Mill) does.

No globals flags, of course :-)  Each value on the belt has its own metadata.

In http://millcomputing.com/topic/metadata/#post-419 (http://millcomputing.com/topic/metadata/#post-419) Ivan wrote:

"It would certainly be possible to have a carry metabit in each byte, and we considered it. After all, there is already a NaR bit in each byte. However, we could not find enough usage for such a design to justify the cost."

That was 2014. Things could have changed. I don't know.


I make the following bold claim: No clean sheet ISA designed for high performance since 1990 has had a carry flag, or global flags register.

Note 1) IBM RS/6000->PowerPC->POWER is exactly 1990. It has 8 flags registers.

Note 2) ARM Aarch64 is not a clean sheet design. While it makes quite a lot of changes from Aarch32, it needs to share an execution pipeline with 32 bit code, with both running optimally, at least for the first 10 years or so.
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 27, 2022, 02:10:46 am
MIPS
DEC Alpha
Intel IA64
RISC-V

Probably others I can't think of right now.

also, subtract with borrow (68k) vs subtract with carry (ppc)

Correct.

At least the following use subtract with carry: IBM System/360, 6502, PIC, MSP430, ARM, PowerPC

At least the following use subtract with borrow: 8080/z80, 8051, x86, M680x, M68k, AVR
Title: Re: A new, hardware "oriented" programming language
Post by: pcprogrammer on November 27, 2022, 05:41:39 am
At least the following use subtract with carry: IBM System/360, 6502, PIC, MSP430, ARM, PowerPC

Don't know about the others but although the 6502 instruction is called SBC (subtract with carry) my R6500 programming manual states it is with borrow.

You have to set the carry before the first subtract to indicate no borrow.
Title: Re: A new, hardware "oriented" programming language
Post by: brucehoult on November 27, 2022, 07:40:29 am
At least the following use subtract with carry: IBM System/360, 6502, PIC, MSP430, ARM, PowerPC

Don't know about the others but although the 6502 instruction is called SBC (subtract with carry) my R6500 programming manual states it is with borrow.

You have to set the carry before the first subtract to indicate no borrow.

Then the manual is naming it incorrectly, at least according to the modern lexicon. Setting carry to indicate no borrow means that borrow is the inverse of the carry.

On the Intel, Motorola etc machines in the other list, you need to clear carry before doing the first word of a subtract -- carry for add and borrow for subtract are in the same sense.

On most of the machines there is a plain SUB instruction as well as SBC, so you never need to be aware of what you would set carry to before the first SBC in a chain. The convention shows up only in the result afterwards: Carry set if there was no borrow in the machines in first list, carry set if there was a borrow in the machines in the second list.

To put it more mathematically:

On all machines, X + Y is implemented as X + Y + C, giving an N+1 bit sum with the MSB going to C

On 6502 etc X - Y is done as X + ~Y + C, , giving an N+1 bit sum with the MSB going to C

On 8080 etc X - Y is done as X + ~Y + ~C, , giving an N+1 bit sum with ~MSB going to C


On 6502, ARM, PowerPC, MSP430 etc, the only difference between add and subtract is inverting Y.

On the other type, the C flag is inverted both before and after a subtraction, complicating the design, adding two inverter delays to the critical path etc.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 27, 2022, 08:58:46 am
If Alpha had crossed my mind, I would have presumed it didn't, since that went to all extremes to excise everything they could. (And, IIRC, excised byte addressing - oops).

Alpha always had byte addressing. What it doesn't have, for the first half dozen models (until 199's "EV56" 21164A), was byte (and 16 bit) load and store instructions. It wasn't a raw speed thing -- Alpha was often the fastest CPU in the world during that time, including on software doing a lot of byte operations.

The problem was multiprocessor systems and software that assumes byte accesses are atomic.

Yes, that's it.

I always admired Alpha's purism, but was skeptical about the lack of byte load/store. The Mill is careful to avoid that excessive purism.

Around the same time I was skeptical about the approach of the HP PA-RISC successor, ItanicItanium. At that time any tiny change to the implementation required someone to re-hand-optimise inner loops; the compiler was supposed to solve that issue, but never quite managed it. In addition, just as CPU power consumption was becoming the limiting factor, the Itanic strategy was to waste power doing lots of speculative execution.

Quote
Quote
I'm not sure what the other currently interesting architecture (the Mill) does.

No globals flags, of course :-)  Each value on the belt has its own metadata.

In http://millcomputing.com/topic/metadata/#post-419 (http://millcomputing.com/topic/metadata/#post-419) Ivan wrote:

"It would certainly be possible to have a carry metabit in each byte, and we considered it. After all, there is already a NaR bit in each byte. However, we could not find enough usage for such a design to justify the cost."

That was 2014. Things could have changed. I don't know.

That sounds right.

One thing I like about the Mill is that its creators have a detailed knowledge of successful radically different hardware and software architectures, how they interact and how each feature meshes with the other features. They then have the good taste to choose wise goals and select/reject features that support the goals.

Basically, since they evolved their understanding at the tight era, they are "renaissance men" of computer system architecture. I doubt we will see people with such a wide knowledge again.

The last time I saw such a good summary of a new product based on selecting/rejecting features based on historical experience was Gosling's Java Whitepaper in 1996.

Quote
I make the following bold claim: No clean sheet ISA designed for high performance since 1990 has had a carry flag, or global flags register.

Note 1) IBM RS/6000->PowerPC->POWER is exactly 1990. It has 8 flags registers.

Note 2) ARM Aarch64 is not a clean sheet design. While it makes quite a lot of changes from Aarch32, it needs to share an execution pipeline with 32 bit code, with both running optimally, at least for the first 10 years or so.

It does seem that way. The flags can only be computed after the operation has been computed. That wasn't a problem when logic/memory speeds were at a certain ratio, but it ceased to be the case in the mid-late 80s.

The other strategy exhibiting that phenomenon was the TI9900's registers being in memory. Great for fast context changes, but somewhat limiting otherwise.
Title: Re: A new, hardware "oriented" programming language
Post by: Siwastaja on November 27, 2022, 12:05:11 pm
I think the unimportance of details in grammar can be demonstrated with a simple example:

Consider the achievements of human kind. We have built pyramids, airplanes, went to the Moon and whatnot. All of this is possible thanks to human being able to communicate with others, with a natural language.

Yet, such great achievements have been made in countless of different cultures, with different languages, with very different grammars and syntax.

It's the process itself, behind the grammar and syntax, which dictates the result. A surgeon can discuss with their colleagues in English, Chinese or Finnish just fine, and as long as they know the language, they absolutely do not care, it simply does not matter to them.

Same can be said about programming languages, for example C and Ada have very different looking grammar and syntax but if you learn both, it's confusing only in the very beginning. You learn the grammar and syntax quickly.

That's why it's more important to start from what the language can do and how, then the grammar will follow.
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 27, 2022, 02:11:44 pm
Around the same time I was skeptical about the approach of the HP PA-RISC successor, ItanicItanium. At that time any tiny change to the implementation required someone to re-hand-optimise inner loops; the compiler was supposed to solve that issue, but never quite managed it. In addition, just as CPU power consumption was becoming the limiting factor, the Itanic strategy was to waste power doing lots of speculative execution.
It was obvious from the very start of Itanium development that its strategy would make for extremely high power consumption. So, it seemed to be a strategy to split the market into mobile and non-mobile threads. That was always a very weird strategy for a company like Intel, when notebooks were a booming business.
The other strategy exhibiting that phenomenon was the TI9900's registers being in memory. Great for fast context changes, but somewhat limiting otherwise.
No architecture is general purpose. They all have a time and a place. The 9900, and the 990 mini-computers based on the same architecture, worked really well in their day. The 990 was one of the biggest selling min-computer lines, although most people have barely heard of it. It crunched a lot of number in things like geophysical survey and defence applications. Memory and circuit performance changed. Even small machines started to get cache. All the underpinings of why the 990 made sense changed, and it went away.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 27, 2022, 03:27:05 pm
concentrate on my questions.
.

We are not here at your beck and call...

Fascinating to contrast that remark with:

Now you need to read (not skim) the points, and do the work...

I suggest you parse your own remarks sir, there's more going on that meets the eye.

If you really can't discuss this subject without continually reverting to discussing me, my perceived needs then please simply refrain.

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 27, 2022, 03:31:46 pm
Your position is pessimistic, discouraging, not open minded, this is nothing to do with your undoubted credentials either, it is you, your hostile tone. This is just a discussion and you're frothing and fuming.

The only frothing is coming at me, not from me.

I see others I respect have now reached and expressed the same conclusion as I did. To their credit, they perhaps gave the benefit of the doubt for longer, though not forever.

Please stop discussing me, that seems all you want to do, discuss me all the time, it's a hallmark of poor reasoning skills. If you were a prosecution lawyer you'd likely lose every case because you can't win a trial by simply attacking the defense lawyer over and over and over.

One must be dispassionate, leverage facts, logic and sound reasoning to reach sound conclusions, all you seem to want to do is berate me, if you disagree with a statement I've made then quote that, and explain - calmly, politely and rationally, why to disagree, it really is that simple, or could be.


Title: Re: A new, hardware "oriented" programming language
Post by: Siwastaja on November 27, 2022, 03:32:06 pm
That's exactly why ATmega32u4 is one of my favourite 8-bitters.  It has native full-speed USB (12 Mbit/s, can do 1 Mbyte/sec even over USB Serial), but still is completely programmable without any HAL or similar, on bare metal, using nothing but avr-gcc (and avr-libc, if you really want), in (freestanding) C or C++.

Now that we are anyway sidetracked (as always in these threads!), I have to add, although I think you know this pretty well but others may not: in my experience almost all microcontrollers can be described by being "completely programmable without any HAL or similar, on bare metal, using nothing but avr-gcc (and avr-libc, if you really want), in (freestanding) C or C++".

All the same old principles and strategies apply. The only thing to do, really, is to ignore advice which claims otherwise and keep doing it exactly like you put it,
just replace "avr-<gcc|binutils|libc>" with "arm-none-eabi-<gcc|binutils|libc>" or whatever architecture you are using.

I mean, I work that way 100%, and it makes jumping between different microcontrollers a breeze, especially important in these times of poor component availability.

My new favorite microcontroller family for bare metal programming is nRF52. The peripheral system is designed ground-up just ease of bare metal programmability in mind. No "turn the peripheral on from multiple places" nonsense, like in most ARM controllers. Dedicating complete 32-bit register for one thing reduces performance a bit, but not much; it's much easier to write PERIPHERAL->TASKS_START = 1; than to work with multi-bit control registers. Or polling PERIPHERAL->EVENTS_RXDONE instead of a bitmask of a status register, you don't need to think about those #defines at all.

Extremely simple configurable task-event routing system: write &PERIPH1->EVENT_XYZ into one register, and &PERIPH2->TASK_ABC into another, and now they are mapped: EVENT_XYZ triggers TASK_ABC without CPU interaction. You can also see they use addresses of registers instead of separate mapping channel numbers. Probably cost them a few dozen logic gates; meaningless when the CPU alone is tens of thousands anyway.

Not much to configure; for example, DMA does not need to be configured from a "DMA controller" at all, as there is no separate "DMA controller" at least externally exposed, and no channels to map. DMA is nothing more than a pointer register plus data count register in the peripheral itself! Here is how you use UART with DMA bare metal in nRF52:

Code: [Select]
uint8_t buffer[] = {0xac, 0xdc, 0xab, 0xba, 0xcd};
NRF_UARTE0->TXD.PTR = (uint32_t)buffer;
NRF_UARTE0->TXD.MAXCNT = sizeof buffer;
NRF_UARTE0->TASKS_STARTTX = 1;

while(!NRF_UARTE0->EVENTS_ENDTX)
;

They do offer classical non-DMA UART, but call it deprecated, because the DMA version is so obvious there is no reason not to use it.

These nRF guys really succeeded making register level programming look like a high-level API! And it's a total polar opposite to the Rapsberry Pi Pico folks, who deliberately made the peripherals as difficult as possible, and decided to force people into using their library code by directly documenting the library use into datasheet. Some may like it, but if you compare it to NRF52, I think it's a total failure. Documenting a library into the datasheet would only make sense if there was some fundamental reason why these peripherals must be difficult, necessitating using complicated code, and then it's good to provide a high-quality, well documented library, and use it throughout all examples. But this is not the case with microcontrollers, as Nordic Semi hardware engineers have well demonstrated; most of the peripherals do fundamentally very simple things.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 27, 2022, 03:39:45 pm
I think the unimportance of details in grammar can be demonstrated with a simple example:

Consider the achievements of human kind. We have built pyramids, airplanes, went to the Moon and whatnot. All of this is possible thanks to human being able to communicate with others, with a natural language.

Yet, such great achievements have been made in countless of different cultures, with different languages, with very different grammars and syntax.

It's the process itself, behind the grammar and syntax, which dictates the result. A surgeon can discuss with their colleagues in English, Chinese or Finnish just fine, and as long as they know the language, they absolutely do not care, it simply does not matter to them.

Same can be said about programming languages, for example C and Ada have very different looking grammar and syntax but if you learn both, it's confusing only in the very beginning. You learn the grammar and syntax quickly.

That's why it's more important to start from what the language can do and how, then the grammar will follow.

Two things.

1. I largely agree with you but that does not demonstrate that all programming language grammars are logically identical, grammars have properties and they can be compared by tabulating those properties, different languages have different properties - surely you must agree with me on this point?

2. There is no algorithm for designing languages no formula, one can consider a grammar for certain interesting properties then jump to looking at semantics and desirable language features. Then one can jump back and alter or enhance the grammar, back and forth, be creative, creativity is not algorithmic there are no rules for creativity, each of has a mind and our minds work in different ways at different times, what matters is not the journey but the destination, there is more than one route.

So of course we can discuss what the language can do, go ahead I've been asking for some nine pages now! For example I recently said that I think the language should support runtime access to metadata like string capacity/lengths, arrays dimensions and bounds and so on. The C language is - IMHO - rather poor in that area, so lets address that.

Title: Re: A new, hardware "oriented" programming language
Post by: Siwastaja on November 27, 2022, 03:59:54 pm
So of course we can discuss what the language can do, go ahead I've been asking for some nine pages now! For example I recently said that I think the language should support runtime access to metadata like string capacity/lengths, arrays dimensions and bounds and so on. The C language is - IMHO - rather poor in that area, so lets address that.

Accessing implementation internals like string capacity/length, but isn't that exactly where C is the best tool you have? Higher level languages hide all such implementation details in their string classes (that's the whole point). In C, such string type does not exist, you have to create your own, which also means you have access to everything you want, and exactly as you want. More work, but this is why embedded developers with limited resources prefer C.

Arrays are indeed a weak point in C, being too primitive and requiring you to babysit implementation details, but if you did actually read any of this thread, it has been discussed in numerous replies above, mostly by Mr. N. Animal. No need to complain about this not being discussed.

On the other hand, people often write crappy code like this, as taught in classes:
Code: [Select]
#define ARRAY_LEN
unsigned char array[ARRAY_LEN] = {1, 2, 3, 4};

for(int i=0; i<ARRAY_LEN; i++)
   ...


When C is much more capable:
Code: [Select]
uint8_t array[] = {1, 2, 3, 4}; // automagic array length!

for(int i=0; i<NUM_ELEM(array); i++) // despite being automagic, length is still known, at compile time! Googling the NUM_ELEM helper macro is left as an excercise for reader. Add to utils.h or something
    ...

Of course still quite primitive, but... We manage! And those who complain, often do not even know what we have available. As I said earlier, use C to its fullest, and it's not as crappy as some lazy examples from 1980's seem like.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 27, 2022, 04:15:41 pm
Around the same time I was skeptical about the approach of the HP PA-RISC successor, ItanicItanium. At that time any tiny change to the implementation required someone to re-hand-optimise inner loops; the compiler was supposed to solve that issue, but never quite managed it. In addition, just as CPU power consumption was becoming the limiting factor, the Itanic strategy was to waste power doing lots of speculative execution.
It was obvious from the very start of Itanium development that its strategy would make for extremely high power consumption. So, it seemed to be a strategy to split the market into mobile and non-mobile threads. That was always a very weird strategy for a company like Intel, when notebooks were a booming business.

Intel did not design the Itanic architecture. HP designed the Itanic.

The objective was simple. In 1989 it was known that HP PA-RISC would run out of steam in a decade's time. When that became significant to customers, they would look for alternatives. What became the Itanic was simply a means of keeping customers buying HP PA-RISC machines. By the mid to late 90s that was the server market.
Title: Re: A new, hardware "oriented" programming language
Post by: Mechatrommer on November 27, 2022, 04:28:14 pm
I've been pondering the idea of attempting to create a new (compiled) programming language specifically designed for hardware programming... tldr...
This is not as crazy as it might at first appear, I'm a very experienced software developer
leave the job to computer scientists and linguists, or at least, go discuss with them. what you are trying to do is like a very experienced engineer who want to lay a new theoritical physics in quantum mechanics. you may ended up reinventing the wheel or may not know what you are dealing with.. if you have to ask here, you are not the man trust me, i'm sorry to break the mood.. i'm not the man, nor most of us here. so rather than keep heated arguing endlessly, like a bunch of experienced engineers discussing about quantum mechanics that they have no clue about... find some reading materials about what a "computer programming language" is all about.. its a classical knowledge nothing new, C/C++ language was designed to be as low level as it can get while maintaining human readable syntax and avoid coupling to certain machine architecture, but generic and compatible enough for all. if you think you can do better, show your working proof. just a blueprint or "list of ideas" will not be enough. try do it actually and you may find the actual barrier in the middle of your journey. just "list of ideas" is easy to brag about. if you have to ask me what features i want in a programming language? i would say none! C/C++ provides me all i need to convert the semantic of procedural processes into machine readable (actual working) processes, through subroutines, definitions etc. if you think you have struggled in embedded system, i doubt your proficiency as a software developer, esp if you cannot differentiate between what a "language" and "compiler" and "machine dependent codes" is? you can make a parser or a compiler not necessarily you can make a new "language". cheers, ymmv.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 27, 2022, 04:40:38 pm
So of course we can discuss what the language can do, go ahead I've been asking for some nine pages now! For example I recently said that I think the language should support runtime access to metadata like string capacity/lengths, arrays dimensions and bounds and so on. The C language is - IMHO - rather poor in that area, so lets address that.

Accessing implementation internals like string capacity/length, but isn't that exactly where C is the best tool you have? Higher level languages hide all such implementation details in their string classes (that's the whole point). In C, such string type does not exist, you have to create your own, which also means you have access to everything you want, and exactly as you want. More work, but this is why embedded developers with limited resources prefer C.

Many high level languages actually let you find the current and maximum lengths of strings or the rank and bound of arrays at runtime. C does not, sure you can write code to somehow create abstractions that do that if you really want, but that's really what languages are for, to reduce the need for such drudgery. If the language can reasonably do something that reduces developer effort then let's exploit that, after all we have "while" and "until" - why have that if we can "simulate" that with a goto loop?


Arrays are indeed a weak point in C, being too primitive and requiring you to babysit implementation details, but if you did actually read any of this thread, it has been discussed in numerous replies above, mostly by Mr. N. Animal. No need to complain about this not being discussed.

On the other hand, people often write crappy code like this, as taught in classes:
Code: [Select]
#define ARRAY_LEN
unsigned char array[ARRAY_LEN] = {1, 2, 3, 4};

for(int i=0; i<ARRAY_LEN; i++)
   ...


When C is much more capable:
Code: [Select]
uint8_t array[] = {1, 2, 3, 4}; // automagic array length!

for(int i=0; i<NUM_ELEM(array); i++) // despite being automagic, length is still known, at compile time! Googling the NUM_ELEM helper macro is left as an excercise for reader. Add to utils.h or something
    ...

Of course still quite primitive, but... We manage! And those who complain, often do not even know what we have available. As I said earlier, use C to its fullest, and it's not as crappy as some lazy examples from 1980's seem like.

I do not dispute that one can "manage" in C, just as one can "manage" in assembler, but the fact that one can manage in assembler does not demonstrate the C language is therefore unnecessary.

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 27, 2022, 04:43:52 pm
Around the same time I was skeptical about the approach of the HP PA-RISC successor, ItanicItanium. At that time any tiny change to the implementation required someone to re-hand-optimise inner loops; the compiler was supposed to solve that issue, but never quite managed it. In addition, just as CPU power consumption was becoming the limiting factor, the Itanic strategy was to waste power doing lots of speculative execution.
It was obvious from the very start of Itanium development that its strategy would make for extremely high power consumption. So, it seemed to be a strategy to split the market into mobile and non-mobile threads. That was always a very weird strategy for a company like Intel, when notebooks were a booming business.

Intel did not design the Itanic architecture. HP designed the Itanic.

The objective was simple. In 1989 it was known that HP PA-RISC would run out of steam in a decade's time. When that became significant to customers, they would look for alternatives. What became the Itanic was simply a means of keeping customers buying HP PA-RISC machines. By the mid to late 90s that was the server market.


Intel's biggest blunder: Itanium. (https://www.youtube.com/watch?v=3oxrybkd7Mo)
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 27, 2022, 04:48:51 pm
I've been pondering the idea of attempting to create a new (compiled) programming language specifically designed for hardware programming... tldr...
This is not as crazy as it might at first appear, I'm a very experienced software developer
leave the job to computer scientists and linguists, or at least, go discuss with them. what you are trying to do is like a very experienced engineer who want to lay a new theoritical physics in quantum mechanics. you may ended up reinventing the wheel or may not know what you are dealing with.. if you have to ask here, you are not the man trust me, i'm sorry to break the mood.. i'm not the man, nor most of us here. so rather than keep heated arguing endlessly, like a bunch of experienced engineers discussing about quantum mechanics that they have no clue about... find some reading materials about what a "computer programming language" is all about.. its a classical knowledge nothing new, C/C++ language was designed to be as low level as it can get while maintaining human readable syntax and avoid coupling to certain machine architecture, but generic and compatible enough for all. if you think you can do better, show your working proof. just a blueprint or "list of ideas" will not be enough. try do it actually and you may find the actual barrier in the middle of your journey. just "list of ideas" is easy to brag about. if you have to ask me what features i want in a programming language? i would say none! C/C++ provides me all i need to convert the semantic of procedural processes into machine readable (actual working) processes, through subroutines, definitions etc. if you think you have struggled in embedded system, i doubt your proficiency as a software developer, esp if you cannot differentiate between what a "language" and "compiler" and "machine dependent codes" is? you can make a parser or a compiler not necessarily you can make a new "language". cheers, ymmv.

Another "I'm content with 'C' so any kind of discussion is fruitless" post, needless to say there are a number of inaccuracies and errant assumptions in what you say, I'll leave it at that.


Title: Re: A new, hardware "oriented" programming language
Post by: Mechatrommer on November 27, 2022, 04:57:15 pm
Many high level languages actually let you find the current and maximum lengths of strings or the rank and bound of arrays at runtime. C does not, sure you can write code to somehow create abstractions that do that if you really want, but that's really what languages are for, to reduce the need for such drudgery.
at what cost? as a language designer, you should already have the answer in your belt. btw, you can start designing one "highly managed" language if you want to, you can start a list of mostly used features and we would love to try... if what you have in mind is that (highly managed language), looking at C is a bad start. start looking at other highly managed language such as maybe Python? or Rust? or whatever, i dont know, i'm not a proficient "all languages" programmer... C is not a managed language. but most people use it. its like automatic car vs manual car. both will have market. i choose manual car... lets see if one day manual car extinct. C#.. anybody?
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 27, 2022, 05:15:12 pm
Many high level languages actually let you find the current and maximum lengths of strings or the rank and bound of arrays at runtime. C does not, sure you can write code to somehow create abstractions that do that if you really want, but that's really what languages are for, to reduce the need for such drudgery.

at what cost? as a language designer, you should already have the answer in your belt.

Well you need to define "cost" first and you haven't. Does it mean man hours to implement in the compiler? memory used for the metadata? performance costs in simply having the metadata, performance cost in accessing the metadata or what? It's a totally valid question and one that will certainly have an answer once we define "cost".

btw, you can start designing one "highly managed" language if you want to, you can start a list of mostly used features and we would love to try... if what you have in mind is that (highly managed language), looking at C is a bad start. start looking at other highly managed language such as maybe Python? or Rust? or whatever, i dont know, i'm not a proficient "all languages" programmer... C is not a managed language. but most people use it. its like automatic car vs manual car. both will have market. i choose manual car... lets see if one day manual car extinct.

I'm certainly not advocating a "managed" language in the sense of C# or Java, I'm looking a compiled imperative language with a grammar borrowed mostly from PL/I but including parts of other grammars too.

At a minimum the language would include:

1. More storage classes than C (defined, based etc)
2. More data types including decimal/BCD and strings of fixed or variable length.
3. Arbitrary rank arrays with optional bound specifiers.
4. Distinguish between function and procedures.
5. Fine control over padding and alignment and field ordering etc.
6. No object oriented features, no GC, no virtual functions and so on.
7. No need for "forward declarations"
8. Nested functions/procedures.
9. Some kind of "computed" goto support.
10. No reserved words.
11. No explicit pointer arithmetic, like ptr++ and ptr-- and so on.
12. Invocable variables (a simpler form of C's function pointer)

That's the basic initial list anyway, entirely reasonable and feasible and something I've implemented successfully in the past.

What I'd like to add is stuff that might be specific to MCU development, features that are not so relevant for conventional programming, (like for example constant execution time functions/procedures where the execution is - or strives to be - independent of state or arguments).

I don't the there's an appreciation of the benefits of a no-reserved-words grammar. We can add new keywords freely and never break backward compatibility.

Look at this draft example:

Code: [Select]

func get_fastest_motors_speed (X) float(binary(128)) optimize(speed) determinate
{
   arg X binary(16);

   return (something);
}


Those keywords decorating the function definition, could grow, we could add new keywords and never break existing code (the various attributes too can specified in any order).

That's just for example, the specific terms used are made up, not really to be taken as definitive, just made up to show the potential flexibility.





Title: Re: A new, hardware "oriented" programming language
Post by: Mechatrommer on November 27, 2022, 05:22:29 pm
Well you need to define "cost" first and you haven't.
"resources" cost, mcu storage space and RAM to store all those drudgery avoidance codes. your language may not run very well in atTiny mcu, or asks for larger ARM processor etc. its inevitable... ;) this will open to endless debate. so just make one and put it into the ring, lets see who takes the bait.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 27, 2022, 05:47:50 pm
Well you need to define "cost" first and you haven't.
"resources" cost, mcu storage space and RAM to store all those drudgery avoidance codes. your language may not run very well in atTiny mcu, or asks for larger ARM processor etc. its inevitable... ;) this will open to endless debate. so just make one and put it into the ring, lets see who takes the bait.

A fixed length string needs (say) 1 or 2 bytes for its capacity.
A varying string needs likewise, 2 or 4 bytes 1 or 2 for capacity and 1 or 2 for current length.

The capacity is a constant so can be shared by all strings of that capacity. That is:

Code: [Select]

dcl usernames(1024) string(64); // array of 1024 strings - total metadata is 2 bytes in this specific example.


Would use only 1 byte of capacity metadata for all 1024 strings. That's a negligible cost in most scenarios I imagine. A similar analysis holds for arrays, small handful of bytes, likely one for rank and two for upper bound and two for lower bound, or perhaps one could optimize that. If the array is declared in such a way one byte will suffice for each bound then we'd do that. You can do a lot of stuff like that in a compiler, one we've generated the AST there is a huge amount of information present that can be used to make decisions like that when code generation starts.

If we adopt PL/I's support for "*" length specifiers, then we can even write code like this:

Code: [Select]

func parse_text (text) bool
{

   arg text string(*);

   L = length(text); // Get the capacity of this fixed length string

}

That function could be called in umpteen places, one might pass a string(32) another a string(1024) another a string(8) etc, the code can at runtime nevertheless find that capacity.

There are other details of course, this isn't a full analysis but you get the idea. In fact one thing emerges and that's that we could have two length oriented builtin functions - "capacity" and "length". (Actually I just looked and the original IBM PL/I had "maxlength" and "length" I think "capacity" is neater!)



Title: Re: A new, hardware "oriented" programming language
Post by: Mechatrommer on November 27, 2022, 05:54:32 pm
thats too optimistic... what about "managerial codes" (framework) to handle all those?
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 27, 2022, 06:08:52 pm
This is the kind of things I enjoy discussing, as I look at the above another optional attribute comes up:

Code: [Select]

dcl usernames(1024) string(64) opaque; // array of 1024 strings - no string metadata

dcl gradenames(1024) opaque string(64) opaque; // array of 1024 strings - no string metadata no array metadata

/* or */

dcl usernames(1024) string(64,opaque); // array of 1024 strings - no string metadata

dcl gradenames(1024,opaque) string(64,opaque); // array of 1024 strings - no string metadata no array metadata



Here we just added a new keyword "opaque" that tells the compiler "Do not retain any metadata for this datum" so that has zero overheads. We could use that on a declaration for some datum that we knew had no need, ever for such metadata, this is just thinking aloud, not saying this is a good or bad idea really. But being able to control whether such metadata is generated or not, is surely the kind of thing that would help if writing code for an MCU, also if such declarations are on the stack of an invoked function, avoiding metadata for small routines would be helpful, there's obviously a cost there that's not necessary if one never needs to know lengths or ranks etc.

 

Title: Re: A new, hardware "oriented" programming language
Post by: Mechatrommer on November 27, 2022, 06:11:50 pm
well thats more and more becoming machine friendly grammar ;D good luck!
Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 27, 2022, 11:03:57 pm
Around the same time I was skeptical about the approach of the HP PA-RISC successor, ItanicItanium. At that time any tiny change to the implementation required someone to re-hand-optimise inner loops; the compiler was supposed to solve that issue, but never quite managed it. In addition, just as CPU power consumption was becoming the limiting factor, the Itanic strategy was to waste power doing lots of speculative execution.
It was obvious from the very start of Itanium development that its strategy would make for extremely high power consumption. So, it seemed to be a strategy to split the market into mobile and non-mobile threads. That was always a very weird strategy for a company like Intel, when notebooks were a booming business.

Intel did not design the Itanic architecture. HP designed the Itanic.

The objective was simple. In 1989 it was known that HP PA-RISC would run out of steam in a decade's time. When that became significant to customers, they would look for alternatives. What became the Itanic was simply a means of keeping customers buying HP PA-RISC machines. By the mid to late 90s that was the server market.
The Itanium's basic design originated at HP, and HPs own silicon designers did much or all of the design for the first silicon. However, Intel bought into this program, not just as a silicon fabricator, but as their core performance strategy going forwards. It seems they were so eager for something completely proprietary to them as their step to 64 bit systems, they forgot all the lessons the industry had learned up to that point - compatibility is king, software moves slowly, so make sure you can run the same software in the maximum possible number of places, old code needs to run well on new hardware, etc. The last point they learned well, when the brilliant Pentium Pro design bombed in the market, because it ran old 16 bot Microsoft code too slowly. Having been burned badly by that, just a few years later they did the same thing, putting a half baked x86 mode into the Itanium.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 27, 2022, 11:13:40 pm
While reviewing the state of fixed and floating point numeric data and types, I came across this (emphasis mine)

Quote
Explicit support for fixed-point numbers is provided by a few computer languages, notably PL/I, COBOL, Ada, JOVIAL, and Coral 66. They provide fixed-point data types, with a binary or decimal scaling factor. The compiler automatically generates code to do the appropriate scaling conversions when doing operations on these data-types, when reading or writing variables, or when converting the values to other data types such as floating-point.

and

Quote
Moreover, in 2008 the International Standards Organization (ISO) issued a proposal to extend the C programming language with fixed-point data types, for the benefit of programs running on embedded processors.[3] Also, the GNU Compiler Collection (GCC) has back-end support for fixed-point.[4][5]

That's from Wikipedia. There's more too (https://gcc.gnu.org/wiki/FixedPointArithmetic):

Quote
Digital Signal Processors have traditionally supported fixed-point arithmetic in hardware. But more recently, many DSP-enhanced RISC processors are starting to support fixed-point data types as part of their native instruction set. When the precision requirements of the application can be met with fixed-point arithmetic, then this is preferred since it can be smaller and more efficient than floating-point hardware. DSP algorithms often represent the data samples and the coefficients used in the computation as fractional numbers (between -1 and +1) to avoid magnitude growth of a multiplication product. Fractional data type, where there are zero integer bits, is a subset of the more general fixed-point data type.

So it seems that explicit programming language support for such data types would be prudent. Additionally, floating point (binary) support for "half" as well as "single" and "double" seems like a no brainer too.

Finally there really needs to be (https://gcc.gnu.org/onlinedocs/gcc/Fixed-Point.html) a more systematic way to represent some things like this:

Quote
The fixed-point types are
short _Fract,
_Fract,
long _Fract,
long long _Fract,
unsigned short _Fract,
unsigned _Fract,
unsigned long _Fract,
unsigned long long _Fract,
_Sat short _Fract,
_Sat _Fract,
_Sat long _Fract,
_Sat long long _Fract,
_Sat unsigned short _Fract,
_Sat unsigned _Fract,
_Sat unsigned long _Fract,
_Sat unsigned long long _Fract,
short _Accum,
_Accum,
long _Accum,
long long _Accum,
unsigned short _Accum,
unsigned _Accum,
unsigned long _Accum,
unsigned long long _Accum,
_Sat short _Accum,
_Sat _Accum,
_Sat long _Accum,
_Sat long long _Accum,
_Sat unsigned short _Accum,
_Sat unsigned _Accum,
_Sat unsigned long _Accum,
_Sat unsigned long long _Accum.

I make no apologies for saying this notation is quite ridiculous, it is retrograde, looking more like some 1940s primitive machine code than a 21st century programming language. Of course if the grammar restricts the flexibility for supporting new data types then perhaps the proponents of this can be forgiven.

This is precisely the kind of stuff I was really hoping to hear more about from engineers here, these are precisely the kinds of expansions of language capabilities that I've been talking about, this is why grammar is so so important, the C grammar is a great example of technical debt that has grown and grown and grown.







Title: Re: A new, hardware "oriented" programming language
Post by: coppice on November 27, 2022, 11:44:46 pm
Quote
The fixed-point types are
short _Fract,
_Fract,
long _Fract,
long long _Fract,
unsigned short _Fract,
unsigned _Fract,
unsigned long _Fract,
unsigned long long _Fract,
_Sat short _Fract,
_Sat _Fract,
_Sat long _Fract,
_Sat long long _Fract,
_Sat unsigned short _Fract,
_Sat unsigned _Fract,
_Sat unsigned long _Fract,
_Sat unsigned long long _Fract,
short _Accum,
_Accum,
long _Accum,
long long _Accum,
unsigned short _Accum,
unsigned _Accum,
unsigned long _Accum,
unsigned long long _Accum,
_Sat short _Accum,
_Sat _Accum,
_Sat long _Accum,
_Sat long long _Accum,
_Sat unsigned short _Accum,
_Sat unsigned _Accum,
_Sat unsigned long _Accum,
_Sat unsigned long long _Accum.

I make no apologies for saying this notation is quite ridiculous, it is retrograde, looking more like some 1940s primitive machine code than a 21st century programming language. Of course if the grammar restricts the flexibility for supporting new data types then perhaps the proponents of this can be forgiven.

This is precisely the kind of stuff I was really hoping to hear more about from engineers here, these are precisely the kinds of expansions of language capabilities that I've been talking about, this is why grammar is so so important, the C grammar is a great example of technical debt that has grown and grown and grown.
I think its time the C standard flipped things around, and started with length specific types, and derived bland vague names, like int and short int, from the specific ones - mostly for backwards compatibility and places where the size of the variable is essentially irrelevant, like a small loop counter.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 28, 2022, 02:37:47 pm
Quote
The fixed-point types are
short _Fract,
_Fract,
long _Fract,
long long _Fract,
unsigned short _Fract,
unsigned _Fract,
unsigned long _Fract,
unsigned long long _Fract,
_Sat short _Fract,
_Sat _Fract,
_Sat long _Fract,
_Sat long long _Fract,
_Sat unsigned short _Fract,
_Sat unsigned _Fract,
_Sat unsigned long _Fract,
_Sat unsigned long long _Fract,
short _Accum,
_Accum,
long _Accum,
long long _Accum,
unsigned short _Accum,
unsigned _Accum,
unsigned long _Accum,
unsigned long long _Accum,
_Sat short _Accum,
_Sat _Accum,
_Sat long _Accum,
_Sat long long _Accum,
_Sat unsigned short _Accum,
_Sat unsigned _Accum,
_Sat unsigned long _Accum,
_Sat unsigned long long _Accum.

I make no apologies for saying this notation is quite ridiculous, it is retrograde, looking more like some 1940s primitive machine code than a 21st century programming language. Of course if the grammar restricts the flexibility for supporting new data types then perhaps the proponents of this can be forgiven.

This is precisely the kind of stuff I was really hoping to hear more about from engineers here, these are precisely the kinds of expansions of language capabilities that I've been talking about, this is why grammar is so so important, the C grammar is a great example of technical debt that has grown and grown and grown.
I think its time the C standard flipped things around, and started with length specific types, and derived bland vague names, like int and short int, from the specific ones - mostly for backwards compatibility and places where the size of the variable is essentially irrelevant, like a small loop counter.

That's a good suggestion. I was refreshing my memory over the IEEE standards and see that defines (among other things) - binary16, binary32, binary64, binary128, decimal32, decimal64 and decimal128.

The binary32 is known as "float" and the binary64 is known as "double", well the less common binary16 is also known as "half" - so these "short hand" names are a convenience.

The .Net ecosystem does this too, it has a systematic technical name for such types and then the less formal convenient name, totally interchangeable. I'm of the opinion that a specification like this is appealing in a new language:

Code: [Select]

...float(dec(32)), float(dec(64)), float(dec(128))
...float(bin(16)), float(bin(32)), float(bin(64)), float(bin(128)), float(bin(256))


Then a subset of these could also have alternative specifiers:

Code: [Select]

float(bin(16)) or half
float(bin(32)) or single
float(bin(64)) or double
float(bin(128)) or quad
float(bin(256)) or oct


Very easy to support grammatically and one could choose whichever specifier is the more appropriate based project/user preferences.








Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 28, 2022, 02:41:50 pm
Ooh, I didn't notice this until now either, suffix for literal constants:

Quote
‘hr’ or ‘HR’ for short _Fract and _Sat short _Fract
‘r’ or ‘R’ for _Fract and _Sat _Fract
‘lr’ or ‘LR’ for long _Fract and _Sat long _Fract
‘llr’ or ‘LLR’ for long long _Fract and _Sat long long _Fract
‘uhr’ or ‘UHR’ for unsigned short _Fract and _Sat unsigned short _Fract
‘ur’ or ‘UR’ for unsigned _Fract and _Sat unsigned _Fract
‘ulr’ or ‘ULR’ for unsigned long _Fract and _Sat unsigned long _Fract
‘ullr’ or ‘ULLR’ for unsigned long long _Fract and _Sat unsigned long long _Fract
‘hk’ or ‘HK’ for short _Accum and _Sat short _Accum
‘k’ or ‘K’ for _Accum and _Sat _Accum
‘lk’ or ‘LK’ for long _Accum and _Sat long _Accum
‘llk’ or ‘LLK’ for long long _Accum and _Sat long long _Accum
‘uhk’ or ‘UHK’ for unsigned short _Accum and _Sat unsigned short _Accum
‘uk’ or ‘UK’ for unsigned _Accum and _Sat unsigned _Accum
‘ulk’ or ‘ULK’ for unsigned long _Accum and _Sat unsigned long _Accum
‘ullk’ or ‘ULLK’ for unsigned long long _Accum and _Sat unsigned long long _Accum

There's a problem here clearly and something really does need to be done about it, band aid and glue just isn't working!

A neater way is required for specifying type info for literal constants...

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 28, 2022, 03:04:46 pm
So here's a question for the MCU engineers:

I am reading lots of articles and blogs about the utility of fixed point arithmetic for certain classes of device. Binary fixed point is quite easy to implement as you know, but what ranges are useful?

The original IBM PL/I language support this generic specifier:

Code: [Select]
dcl counter fixed bin (P,S);

Here P is the total number of digits and S is the "scale factor".

Within implementation defined maximums, any value for P and S was supported.

One could create variables like this:

Code: [Select]
dcl counter1 fixed bin (18,7); // precision=18, scale=7
dcl counter2 fixed bin (29,3);
dcl counter2 fixed bin (30,-6);

Furthermore, such a datum could be declared with a scale/precision that was an expression, not just a compile time constant, not a hard thing to support.

But what kind of flexibility is really needed? could one solve 99% of problems with just a few precisions like 8, 16 and 32?

What about the signedness? Would a rule that even precisions are always unsigned and odd precisions are always signed, work?

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 28, 2022, 04:22:03 pm
I'd be interested in hearing about some real world use cases for coroutines, a couple of people have raised this in this thread.

There isn't (or wasn't ten years ago when I was blogging about coroutines in C#) a lot out there about coroutines, it seems they had their roots in assembler programming (in fact in a particular processors architecture in the 1950s or 1960s, a very specific instruction if I recall...).

So if anyone has coded these in C or assembler, I'd be fascinated to learn more about their real world relevance.

I can see, the concept was first put to use on a Burroughs 220 in 1963.

Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 28, 2022, 06:17:54 pm
Coroutines are rather interesting, looking at the machine code implementation is fascinating from a high level language perspective.

There is some overlap of meanings though in the literature, for example the basic coroutine described in the 1963 paper is not quite the same idea as the more recent iterator pattern and some richer examples of "coroutines" in languages like Kotlin or libraries like Unity are closer to async/await asynchronous enumerables with implied use of multiple threads, there are similarities of course but they are distinct mechanisms.

This simple image captures the essence of what's described in the 1963 paper.

(https://softpanorama.org/Lang/Asmorama/Images/CH19A1.gif)

So as far as I can tell that's the "classic" definition of a coroutine, a mechanism where control can flow as shown and where each procedure retains access to its stack frame as control cycles back and forth.

Of course one can have loops rather than simple linear sequences and there could be three, four or more procedures participating...

So how is that pattern leveraged in MCU work? what kinds of problems can benefit from such a mechanism?



Title: Re: A new, hardware "oriented" programming language
Post by: rstofer on November 28, 2022, 06:23:14 pm
Interesting presentation on the difference between a coroutine and a thread:

https://www.educba.com/coroutines-vs-threads/ (https://www.educba.com/coroutines-vs-threads/)

If the language is going to be used for IoT or Edge computing, it might be helpful to have parallel computing as part of the language.  Fortran already has this feature as part of the language specification.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 28, 2022, 06:40:01 pm
Interesting presentation on the difference between a coroutine and a thread:

https://www.educba.com/coroutines-vs-threads/ (https://www.educba.com/coroutines-vs-threads/)

If the language is going to be used for IoT or Edge computing, it might be helpful to have parallel computing as part of the language.  Fortran already has this feature as part of the language specification.

That's also an informative page, very helpful.

By parallelism are you thinking of multiple cores? or preemptive multitasking on a single core?
Title: Re: A new, hardware "oriented" programming language
Post by: SiliconWizard on November 28, 2022, 06:56:08 pm
Coroutines are a pretty old concept. Modula-2 introduced them in 1978.

You can find them in one form or another in a number of more recent languages. Even scripting languages such as Lua.
Title: Re: A new, hardware "oriented" programming language
Post by: rstofer on November 28, 2022, 07:31:14 pm
By parallelism are you thinking of multiple cores? or preemptive multitasking on a single core?

More as Nvidia style of parallelism across multiple thousand Cuda units with a coordinated completion.  And this explanation from Oracle:

https://docs.oracle.com/cd/E19957-01/805-4940/6j4m1u7qk/index.html

There's a reason I bought a laptop with an RTX3070 GPU and it's the 5120 Cuda units for parallel computing of matrix operations for Machine Learning.  That's a bunch of parallelism if it can be expressed in the language.  And it's just a modest GPU, there are bigger devices as well as standalone units.

ML is alright as a theory but it eventually needs to flow down to the factory floor and that's where the idea of IoT or Edge computing comes into play and things like image recognition tend to take a ton of math.
Title: Re: A new, hardware "oriented" programming language
Post by: rstofer on November 28, 2022, 08:14:49 pm
It seems to me that the multiplicity of numeric types will ultimately lead to a trainwreck.

One thing that I find problematic with Python is that variables are not declared, they just happen.  Given a variable name,  you have no idea what is being represented.  Could be a string, a float, an int, probably a vector, maybe a matrix and even if you thought you knew the shape, even that could change during execution.  How far back do you have to look to find where it was last modified?

Same thing with MATLAB for that matter. 

One thing I like about Modern Fortran is 'implicit none'.  Every variable has to be declared and no more of the leading character determining the type unless otherwise declared. 

I also like the 'intent' attribute:

http://www.personal.psu.edu/jhm/f90/statements/intent.html (http://www.personal.psu.edu/jhm/f90/statements/intent.html)

I'm not sure what to think about functions returning multiple values and the ability to ignore pieces of the return values.  Returning two ndarrays and only keeping one seems bizarre.  But so does using indentation as a syntactic element;

I do like the idea of slicing arrays and being able to concatenate rows or columns.

Although I grumble about the lack of declarations, I do like Python's ndarray.

Is white space going to be significant?  At least in Fortran IV, it wasn't.  The statement:
DO10I=1,4
could be the beginning of a loop or just another real variable being set to a value all the way up to the comma.  I never tested that concept but it was a side effect of the fact that Fortran ignored white space.  Apparently in fixed format Modern Fortran, white space is still ignored but in free form it is significant

https://community.intel.com/t5/Intel-Fortran-Compiler/spaces-not-ignored-in-free-format/td-p/1112317 (https://community.intel.com/t5/Intel-Fortran-Compiler/spaces-not-ignored-in-free-format/td-p/1112317)

Embedded spaces (or under scores) in long numeric strings can be useful.
SpeedOfLight = 186_000 miles per second

Title: Re: A new, hardware "oriented" programming language
Post by: Mechatrommer on November 28, 2022, 08:25:21 pm
One thing that I find problematic with Python is that variables are not declared, they just happen.  Given a variable name,  you have no idea what is being represented.  Could be a string, a float, an int, probably a vector, maybe a matrix and even if you thought you knew the shape, even that could change during execution.  How far back do you have to look to find where it was last modified?

Same thing with MATLAB for that matter. 

One thing I like about Modern Fortran is 'implicit none'.  Every variable has to be declared and no more of the leading character determining the type unless otherwise declared.
my old Basic language has this feature, i ended up enforcing "Option Explicit" everytime so i know what i'm coding. in my dictionary, "Option Implicit" and the things you mentioned about returning multiple values is just good recipe for bloatness. btw i think you are not in love with Python, you are in love with its library ;) cheers.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on November 28, 2022, 09:32:07 pm
It seems to me that the multiplicity of numeric types will ultimately lead to a trainwreck.

One thing that I find problematic with Python is that variables are not declared, they just happen.  Given a variable name,  you have no idea what is being represented.  Could be a string, a float, an int, probably a vector, maybe a matrix and even if you thought you knew the shape, even that could change during execution.  How far back do you have to look to find where it was last modified?

Yes, Python is dynamically typed, types are determined at and verified at runtime, has benefits but also non trivial costs as you've found. Interpreted languages often have dynamic typing for ease of use.

Same thing with MATLAB for that matter. 

APL too, powerful but has its downsides.

One thing I like about Modern Fortran is 'implicit none'.  Every variable has to be declared and no more of the leading character determining the type unless otherwise declared. 

I also like the 'intent' attribute:

http://www.personal.psu.edu/jhm/f90/statements/intent.html (http://www.personal.psu.edu/jhm/f90/statements/intent.html)


Yes, that "intent" is potentially very useful indeed, with sensible use it can reduce subtle errors.

I'm not sure what to think about functions returning multiple values and the ability to ignore pieces of the return values.  Returning two ndarrays and only keeping one seems bizarre.  But so does using indentation as a syntactic element;

Some of these ideas emanate from functional language theory and mathematics, "tuples" for example comes from that world. In such languages they feature very naturally too. C# now has tuples, they are pretty useful because if one wants to return several things then prior to tuples you had to define a struct or class just to wrap these elements. I find them useful in that situation, the need to return multiple - often disparate - things in ways that were not initially anticipated.

I do like the idea of slicing arrays and being able to concatenate rows or columns.

Yes, I agree, there are some things that can be done regarding "slices" that incur very little runtime cost. Some of the strengths of Fortran were carried over into PL/I by the language designers, that too has some nifty ways of dealing with arrays, stuff not seen explicitly in C or C++.

Although I grumble about the lack of declarations, I do like Python's ndarray.

Is white space going to be significant?  At least in Fortran IV, it wasn't.  The statement:
DO10I=1,4
could be the beginning of a loop or just another real variable being set to a value all the way up to the comma.  I never tested that concept but it was a side effect of the fact that Fortran ignored white space.  Apparently in fixed format Modern Fortran, white space is still ignored but in free form it is significant

https://community.intel.com/t5/Intel-Fortran-Compiler/spaces-not-ignored-in-free-format/td-p/1112317 (https://community.intel.com/t5/Intel-Fortran-Compiler/spaces-not-ignored-in-free-format/td-p/1112317)

Embedded spaces (or under scores) in long numeric strings can be useful.
SpeedOfLight = 186_000 miles per second

I've not looked much at all at Python myself. It seems it lets us create arrays whose rank isn't known until runtime, and I can see the convenience of that for some domains and in an interpreted language. Of course arrays are (or can be) just contiguous blocks of memory with a function that can convert n subscripts into an offset into the array, so in that sense they are illusory, just a "way" of perceiving how data is organized.

Spaces? yes I have heard of Fortran's tolerance of that, I guess some of that goes back to the nature of the industry at the time, keyboards were scarce, most people then (including me even in 1982) wrote code onto coding sheets, those had to be transcribed by a team of "punch" operators and so on.

So it's likely Fortran tried to be flexible because of these limited input methods. The grammar I have in mind, borrows from PL/I in several respects. PL/I was designed with a thorough analysis of Fortran, Cobol and Algol, taking some of the best ideas in those and developing a uniform grammar that captured these varied concepts.

Spaces can't be totally ignored, it all depends on what it takes to recognize a language token. Some tokens are simple enough to never be ambiguous, but other are not, like identifiers, these can be arbitrary and any length.

Like Fortran though, PL/I and the grammar I've been exploring, is free of reserved words, like Fortran an identifier can be anything even a keyword, and the grammar rules make it straightforward to resolve.

Here's an interesting historical article (https://softpanorama.org/Lang/pl1.shtml) that sheds some little known light on how Fortran and Cobol and Algol served as the basis for PL/I.

Note:

Quote
Still it was the first language that contained decent string handling capabilities, pointers, three types of allocation of storage -- static, automatic (stack-based) and controlled (heap-based), exception handling, and rudimentary multitasking.  While the idea of preprocessor (a front end macro generator)  was never cleanly implemented (as PL/1 did not stored "macro source" line numbers and without them matching preprocessor and "real statements" was difficult)  it was also innovative and later was inherited and expanded by C.

All-in-all PL/1 was and probably still is one of the most innovative programming languages in existence and some of its feature are still not matched by "new kids" in the programming language block.

Pointers, ->, while/until, break, the /* */ comments, exceptions, multitasking in the language, a powerful preprocessor, semicolon as statement terminator and more, all originated in PL/I and some were carried over into C.












Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on November 28, 2022, 09:36:16 pm
It seems to me that the multiplicity of numeric types will ultimately lead to a trainwreck.

One thing that I find problematic with Python is that variables are not declared, they just happen.  Given a variable name,  you have no idea what is being represented.  Could be a string, a float, an int, probably a vector, maybe a matrix and even if you thought you knew the shape, even that could change during execution.  How far back do you have to look to find where it was last modified?

That can work nicely. If you need to keep a tight rein on the type, then encapsulate that type in a class and control all arithmetic operations.

Quote
Is white space going to be significant?  At least in Fortran IV, it wasn't.  The statement:
DO10I=1,4
could be the beginning of a loop or just another real variable being set to a value all the way up to the comma.  I never tested that concept but it was a side effect of the fact that Fortran ignored white space.  Apparently in fixed format Modern Fortran, white space is still ignored but in free form it is significant

IIRC that lead to spectacular unfortunate consequences.

C used to have a+= b giving the same result as a =+ b  but a=+b was ambiguous.
Title: Re: A new, hardware "oriented" programming language
Post by: rstofer on November 28, 2022, 09:51:57 pm
btw i think you are not in love with Python, you are in love with its library ;) cheers.

That is correct, I enjoy the application specific libraries but I'm not a huge fan of the language.  It'll probably grow on me.  I'm stuck in the Fortran/C world with no intention of upgrading.

For no particularly good reason, I find myself interested in Machine Learning.  It's not like I will ever have another job or otherwise derive monetary value from the study but  I admire the math and how the library functions do things like backpropogation and partial derivatives.
Title: Re: A microcontroller programming language
Post by: rstofer on November 28, 2022, 10:33:36 pm
I would have loved Algol but we didn't have it on the machines I was using circa '70.  Fortran or RPG was about all we could get.  No worries, a couple of assembly language libraries and we had string functionality in Fortran.  IBM provided the Business Subroutines package which we improved locally.

Next high level language up was Pascal and I was in love.  It just looks pretty on the listing.  Most important to me was the concept of nested procedures/functions.  When I found out that C didn't have that feature I was ready to throw rocks at it.  I also like Wirth's syntax diagrams and the way he structure the language such that a recursive descent compiler was all that was needed.  I'm big on recursive descent and nested procedures/functions.  "Algorithms + Data Structures = Programs" by Niklaus Wirth is my favorite book on programming.  The PL0 compiler in the back of the book shows just how simple it is to create a recursive descent compiler.  Alas, it is nearly impossible to pass arbitrary arrays with Pascal and reshaping an array would be right out the window.  I don't know where it stands with Oberon.

I pretty much like all of the languages that are derived from Algol.  I used PL/I in grad school to write an 8080 assembler (this was in '75, things hadn't evolved very far and free assemblers weren't all over the place).  I regret not having another semester to really learn the language.  My advisor thought the code looked a lot like Fortran.  Deadlines...

After it was no longer relevant, I got a copy of PL/M - it is written in Fortran!

Digital Research (CP/M) created a PL/I compiler for the z80/8080 world.  It works well but I haven't used it for anything serious.  I did use Microsoft's Fortran compiler.  I also works well.



Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 28, 2022, 10:56:14 pm
I would have loved Algol but we didn't have it on the machines I was using circa '70.  Fortran or RPG was about all we could get.  No worries, a couple of assembly language libraries and we had string functionality in Fortran.  IBM provided the Business Subroutines package which we improved locally.

Next high level language up was Pascal and I was in love.  It just looks pretty on the listing.  Most important to me was the concept of nested procedures/functions.  When I found out that C didn't have that feature I was ready to throw rocks at it.

Oh I understand that. Pascal and PL/I use nested procedures without a second thought, they are totally natural ways to express things. An inner procedure has access too, to all of the parent procedures automatic variables and parameters, all very elegant and sensible. C# recently added these (https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/local-functions) to the language.

I also like Wirth's syntax diagrams and the way he structure the language such that a recursive descent compiler was all that was needed.  I'm big on recursive descent and nested procedures/functions.  "Algorithms + Data Structures = Programs" by Niklaus Wirth is my favorite book on programming.  The PL0 compiler in the back of the book shows just how simple it is to create a recursive descent compiler.  Alas, it is nearly impossible to pass arbitrary arrays with Pascal and reshaping an array would be right out the window.  I don't know where it stands with Oberon.

Early parsers were recursive descent, then came the automated parsers made from PDAs, fascinating machines but very frustrating due to it being hard to separate concrete parsing from abstract parsing and report errors intelligibly, messages like "The attribute "static" must not appear more than one in a declaration" are so much friendlier than "Unrecognized state encountered while parsing<attribute-list> when parsing <local-declarations> when parsing <procedure-block> when parsing..."

So most newer languages are parsed with hand crafted recursive descent parsers, here's the one I developed for PL/I (https://github.com/Steadsoft/PLI-2000/blob/main/Compiler%20Source/PASS1.C) on Windows some years back - all written in C.

I pretty much like all of the languages that are derived from Algol.  I used PL/I in grad school to write an 8080 assembler (this was in '75, things hadn't evolved very far and free assemblers weren't all over the place).  I regret not having another semester to really learn the language.  My advisor thought the code looked a lot like Fortran.  Deadlines...

After it was no longer relevant, I got a copy of PL/M - it is written in Fortran!

Digital Research (CP/M) created a PL/I compiler for the z80/8080 world.  It works well but I haven't used it for anything serious.  I did use Microsoft's Fortran compiler.  I also works well.

Strange old world sometimes!

I love the story about Steve Jobs telling Seymour Cray that Apple had just purchased a CRAY to help to design the next Apple, whereupon Cray says "Oh really? we just ordered an Apple to help us design the next CRAY".



Title: Re: A microcontroller programming language
Post by: brucehoult on November 28, 2022, 11:35:36 pm
Next high level language up was Pascal and I was in love.  It just looks pretty on the listing.  Most important to me was the concept of nested procedures/functions.  When I found out that C didn't have that feature I was ready to throw rocks at it.

I thought so at the time too, but in retrospect it's a half-arsed feature that is bad for the same reason global variables are bad.

Sure, it's slightly handy for creating comparison functions to pass to sort(), but there are few other good use-cases, and it puts the overhead of maintaining a static link chain or "display" into every single function call and return, for a feature that is seldom used.

In Pascal any variables used by the nested function have to be declared right up at the top of the function, with the nested function up there too, and then the actual use is buried who knows how far down in the body of the function.

In C, you put such comparison functions just before the outer function header, instead of just after, and maybe define a small struct to hold variables (copies or pointers to them) shared between the caller and the "nested" function. The caller needs to allocate the struct, copy a couple of things into it, then pass it as an explicit void* parameter to the sort function. The comparison function needs to cast the void* to the correct type.

C seems like more work, but in fact it is work that Pascal has to do anyway, it's just hidden. In fact, every Pascal I've used did it another way -- the previously mentioned static chains or display -- but they SHOULD have done it the same way you manually do it in C, so that the overhead exists only when you actually use the feature, not in every single function call.

The Pascal feature is just pure sugar, not anything fundamental.

What IS useful and much better than C is when you can write the comparison function right there as an anonymous function in the sort function argument, and it can access variables local to the block it is in, not only top level variables of the enclosing function.  Even if, as in PL/1, you have to give the nested function a name and write it as a separate statement on the line before the call to the sort() function, that's still fine.

The other annoying part about how you do it in C is having to cast from a void* inside the "nested" function. This is avoided in C++, Java etc by making sort() take a pointer to an object of some class ("comparable"?) and making your compare function a member of a subclass derived from it.

I seem to recall Oberon didn't have classes but had "extensible types", including records.

Full closures, that you can return from the enclosing function, or store into global data structures, are off course much more useful than mere nested functions. That is something you can do in plain C simply by heap-allocating that little struct you copy the shared local variables into.
Title: Re: A microcontroller programming language
Post by: Mechatrommer on November 29, 2022, 12:01:28 am
first i learnt fortran in dos in formal course, on 1st day i quickly in love with "computer programming language", then i introduced myself to Delphi/Pascal... C/C++ and Basic.. i left pascal and fortran behind, i keep c and basic until today... both basic and pascal dont support pointer/tree data structure natively... but Basic (VB) is everything i need to build GUI Windows and connect easily to C built Win32 API. C is about "speed" and "very broad" application/hackability/castability. if any of modern/managed languages can beat that, tell me i'm willing to give an eye.. for me they are only worth understanding so i can port their codes back into C...
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 29, 2022, 12:02:23 am
Next high level language up was Pascal and I was in love.  It just looks pretty on the listing.  Most important to me was the concept of nested procedures/functions.  When I found out that C didn't have that feature I was ready to throw rocks at it.

I thought so at the time too, but in retrospect it's a half-arsed feature that is bad for the same reason global variables are bad.

Sure, it's slightly handy for creating comparison functions to pass to sort(), but there are few other good use-cases, and it puts the overhead of maintaining a static link chain or "display" into every single function call and return, for a feature that is seldom used.

In Pascal any variables used by the nested function have to be declared right up at the top of the function, with the nested function up there too, and then the actual use is buried who knows how far down in the body of the function.

In C, you put such comparison functions just before the outer function header, instead of just after, and maybe define a small struct to hold variables (copies or pointers to them) shared between the caller and the "nested" function. The caller needs to allocate the struct, copy a couple of things into it, then pass it as an explicit void* parameter to the sort function. The comparison function needs to cast the void* to the correct type.

C seems like more work, but in fact it is work that Pascal has to do anyway, it's just hidden. In fact, every Pascal I've used did it another way -- the previously mentioned static chains or display -- but they SHOULD have done it the same way you manually do it in C, so that the overhead exists only when you actually use the feature, not in every single function call.

The Pascal feature is just pure sugar, not anything fundamental.

What IS useful and much better than C is when you can write the comparison function right there as an anonymous function in the sort function argument, and it can access variables local to the block it is in, not only top level variables of the enclosing function.  Even if, as in PL/1, you have to give the nested function a name and write it as a separate statement on the line before the call to the sort() function, that's still fine.

The other annoying part about how you do it in C is having to cast from a void* inside the "nested" function. This is avoided in C++, Java etc by making sort() take a pointer to an object of some class ("comparable"?) and making your compare function a member of a subclass derived from it.

I seem to recall Oberon didn't have classes but had "extensible types", including records.

Full closures, that you can return from the enclosing function, or store into global data structures, are off course much more useful than mere nested functions. That is something you can do in plain C simply by heap-allocating that little struct you copy the shared local variables into.

Well yes there are use cases, but as to whether they're good or few or not is dependent upon the problem at hand. The Microsoft C# team supported the introduction of this for several reasons, you can read (https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/local-functions#local-functions-vs-lambda-expressions) about the differences between local function and lambdas/anonymous here, sure it C# but it's largely relevant.

In particular note this important detail - iterators  (https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/local-functions#usage-of-the-yield-keyword)- defining lambdas that are also iterators (are able to yield results) is not supported in C# for a whole variety of reasons.

IMHO these nested procedure is very easy to implement, and ideal for certain scenarios as you said, but I don't see why you equate them to "global" variables, as being as bad as them, what do you mean? If the function were not enclosed yet needed access to the callers state, how would that be done? pass in a load of extra arguments? seems like a lot of extra baggage surely?

See also this discussion: https://stackoverflow.com/a/40949214

Finally:

Quote
In addition to svick's great answer there is one more advantage to local functions:
They can be defined anywhere in the function, even after the return statement.

Pascal did limp a little with its cumbersome insistence on "one pass" (a meaningless term these days anyway) and the consequent impact on parsing and analysis (same in C) so in a new language it would be easy to do as C# does and let you lexically position the function definition anywhere within the containing block's body.


Title: Re: A microcontroller programming language
Post by: SiliconWizard on November 29, 2022, 12:11:18 am
I don't particularly care for nested functions - I remember other threads talking about that as well.
Apart from what you said, that just leads to more visual clutter which is bad. You quickly end up not seeing what exactly is the function being defined and what is not. Functions must be easier to read, not harder.

The underlying problem it's trying to solve is a problem of scope and namespaces. Which can/could be solved using other, more readable approaches.

And helper functions can actually often be reused, so defining them strictly as nested prevents you from any reuse. This is bad. I won't count how often I have been able to reuse helper functions. If you nest them (or make them anonymous, see below), not only can you not reuse them - so you'll be tempted/forced to duplicate them if the same functions are needed elsewhere - but you also cannot unit-test them separately, which can be just as bad. Very much a recipe for bad coding practice.

For the same reason (and also that of readability), I don't care for anonymous functions, expect in very, very limited cases, which IMHO do not make them worth the trouble.

Now those that like Pascal - already mentioned that, but feel free to try freepascal and Lazarus. Great stuff if you like that. Definitely underrated.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on November 29, 2022, 01:40:54 am
In C, you put such comparison functions just before the outer function header, instead of just after, and maybe define a small struct to hold variables (copies or pointers to them) shared between the caller and the "nested" function. The caller needs to allocate the struct, copy a couple of things into it, then pass it as an explicit void* parameter to the sort function. The comparison function needs to cast the void* to the correct type.
A good example of this is the GNU extension to the qsort() function, qsort_r() (https://man7.org/linux/man-pages/man3/qsort_r.3.html).  The qsort_r() takes an additional void pointer, that is passed to the comparison function as the third parameter.  In particular, if your array entries contain an array of potential keys, you can provide the key index via that extra argument (via (void *)(uintptr_t)index and (uintptr_t)pointer casts).  Alternatively, you can use a single comparison function per key type, and pass the byte offset to the key within each structure via that extra argument (using offsetof()).

I for one use closures –– structures describing the state of a function or operation ––, a lot: in C for example, where one might use nested functions or coroutines in other languages.  A typical example is a tokenizing input buffer, a generator of sorts, where one call returns the next field in the current record, and another call skips to the start of the next record, used for example for CSV file parsing.  Queues and FIFOs are another use case, when one side may be generating or consuming more data than may fit in RAM, so the two do need to be interleaved to keep the temporary memory requirements within reason.

A nested function is simply one that receives a closure describing the state of its parent – including any variables it needs access to.  This works even when the 'nested' function is recursive.  Furthermore, such a 'nested' function can be 'nested' under any parent function that uses the same type of closure to describe its state.  So no, I don't really need nested functions either, as long as using such closures/state structures is easy and straightforward.

sizeof, typeof, compatible_types, and typenameof operators should be provided even by statically types languages.
C provides only the first one; GCC, Clang, Intel CC and others provide the two next ones as extensions to C; and the fourth one can only be approximated by preprocessor stringification.  This leads to less than optimal interfaces when values of runtime-dependent types need to be passed.  Typically, the calls need to explicitly specify the type as a literal (token or string in C) and the size of the value, but there is no good way to check they match the expression being passed.  (If the expression type is a structure with a C99 flexible array member, even sizeof won't work.)
In particular, stringifying any typeof (expression) will only lead to a string containing that text, not the type name.

These kinds of use cases are why I believe so strongly that looking at existing code (in different competing languages, suitable for the target niche) to understand what kind of approaches they use for solving these kinds of problems, is extremely important.  Because there is a lot of bad code out there, looking for consensus is not a good idea: you need to find the 'best' ways of solving each kind of a problem, and learn from that.  (To me, 'best' here means robust with errors easy to catch, without imposing a strict usage pattern; and not 'me like most'.)
Title: Re: A microcontroller programming language
Post by: rstofer on November 29, 2022, 02:43:14 am
If you look at the Pascal syntax diagram you will start to see where recursive descent was a part of the syntax and that nested procedures are the way to go:

Syntax Diagrams
https://link.springer.com/content/pdf/bbm:978-1-4757-1764-8/1.pdf

P4 Pascal Compiler Source:
https://homepages.cwi.nl/~steven/pascal/pcom.p

Sets are another underappreciated concept.  They are terrific as used in this compiler.  We have a set of symbols that can start the next statement and a set of symbols that can end the statement (or continue parsing).  At all times, the compiler knows exactly what to look for so error detection and notification are pretty easy.  The heart of the compiler is at 'procedure statement' and the obvious use of nested procedures is in evaluating expressions at 'procedure expression'.

Here are the sets the compiler actually uses:

Code: [Select]
constbegsys,simptypebegsys,typebegsys,blockbegsys,selectsys,facbegsys,
statbegsys,typedels: setofsys;

I spent a lot of time with the source to UCSD Pascal back around 1980.

The source is available for an interpreter:
https://homepages.cwi.nl/~steven/pascal/pint.p

At one time I was working on an FPGA project to implement the interpreter.  I got hung up when I had trouble with the idea of system calls for things like floating point but the rest of it worked.
Title: Re: A microcontroller programming language
Post by: rstofer on November 29, 2022, 03:11:15 am
Now those that like Pascal - already mentioned that, but feel free to try freepascal and Lazarus. Great stuff if you like that. Definitely underrated.

I have fpc (preepascal) installed on all of my Linux boxes.

I don't do much application programming so I don't use it nearly as much as I would like but I do install it on every machine.


Title: Re: A microcontroller programming language
Post by: brucehoult on November 29, 2022, 03:32:28 am
If you look at the Pascal syntax diagram you will start to see where recursive descent was a part of the syntax and that nested procedures are the way to go:

I don't see the link.

Mutually-recursive procedures, yes. But not nested.
Title: Re: A microcontroller programming language
Post by: DiTBho on November 29, 2022, 10:40:55 am
The underlying problem it's trying to solve is a problem of scope and namespaces. Which can/could be solved using other, more readable approaches.

That is a big problem to solve!
Title: Re: A microcontroller programming language
Post by: DiTBho on November 29, 2022, 10:46:12 am
sizeof, typeof, compatible_types, and typenameof operators

- sizeof()
- typeof()

What about the other two operators? Where are they useful in C?
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 29, 2022, 03:13:36 pm
If you look at the Pascal syntax diagram you will start to see where recursive descent was a part of the syntax and that nested procedures are the way to go:

I don't see the link.

Mutually-recursive procedures, yes. But not nested.

I'm interested in exploring the ARM assembly programming, I was reading about it yesterday and the contrast with the crippled x86 ISA is huge. Reading about ARM was reminiscent (for me) of the 68000 which was a very nice device.

It's different of course, the distinction between LOAD/STORE approach and other MCU architectures is important and not something I've ever thought about in detail. I was also quite taken by the fact that the endianness can be altered by an instruction, never seen that before, granted it's restricted but that's not any surprise.

So I have a question, I use Windows as my desktop on (on an AMD CPU) and routinely use Visual Studio with VisualGDB (https://visualgdb.com/) to plat with MCUs. If I want to write and run and debug assembler and I have STM32 devices, what would you suggest? VisualGDB is a good quality tool (certainly for amateur use like mine) but seems to implicitly assume C and doesn't seem to offer me a choice of Assembler when creating a project.

I might be able to just include an assembler source file into a simple project, I read that it will likely understand that, but I wanted to ask someone more expert than me, perhaps there are other tools? other ways? I use VisualGDB because I routinely work with Visual Studio Enterprise for my professional work and very familiar with it and it's a superb system as well, so VisualGDB was a huge appeal.

Thanks in advance.






Title: Re: A microcontroller programming language
Post by: rstofer on November 29, 2022, 03:45:09 pm
If you look at the Pascal syntax diagram you will start to see where recursive descent was a part of the syntax and that nested procedures are the way to go:

I don't see the link.

Mutually-recursive procedures, yes. But not nested.

Try this:
Code: [Select]
procedure expression;
  var lattr: attr; lop: operator; typind: char; lsize: addrrange;

  procedure simpleexpression(fsys: setofsys);
    var lattr: attr; lop: operator; signed: boolean;

    procedure term(fsys: setofsys);
      var lattr: attr; lop: operator;

      procedure factor(fsys: setofsys);
var lcp: ctp; lvp: csp; varpart: boolean;
    cstpart: setty; lsp: stp;
      begin
if not (sy in facbegsys) then
  begin error(58); skip(fsys + facbegsys);
    gattr.typtr := nil

Page 138 and 139 of the syntax diagrams show the relationship between expression, simpleexpression, term and factor.  Note that factor can recursively call expression and start the entire tree over again.

The compiler code flows directly from the syntax diagrams.

Compared to the transition matrix we were taught in grad school, Wirth's approach looks elegant.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 29, 2022, 03:56:54 pm
In C, you put such comparison functions just before the outer function header, instead of just after, and maybe define a small struct to hold variables (copies or pointers to them) shared between the caller and the "nested" function. The caller needs to allocate the struct, copy a couple of things into it, then pass it as an explicit void* parameter to the sort function. The comparison function needs to cast the void* to the correct type.
A good example of this is the GNU extension to the qsort() function, qsort_r() (https://man7.org/linux/man-pages/man3/qsort_r.3.html).  The qsort_r() takes an additional void pointer, that is passed to the comparison function as the third parameter.  In particular, if your array entries contain an array of potential keys, you can provide the key index via that extra argument (via (void *)(uintptr_t)index and (uintptr_t)pointer casts).  Alternatively, you can use a single comparison function per key type, and pass the byte offset to the key within each structure via that extra argument (using offsetof()).

I for one use closures –– structures describing the state of a function or operation ––, a lot: in C for example, where one might use nested functions or coroutines in other languages.  A typical example is a tokenizing input buffer, a generator of sorts, where one call returns the next field in the current record, and another call skips to the start of the next record, used for example for CSV file parsing.  Queues and FIFOs are another use case, when one side may be generating or consuming more data than may fit in RAM, so the two do need to be interleaved to keep the temporary memory requirements within reason.

A nested function is simply one that receives a closure describing the state of its parent – including any variables it needs access to.  This works even when the 'nested' function is recursive.  Furthermore, such a 'nested' function can be 'nested' under any parent function that uses the same type of closure to describe its state.  So no, I don't really need nested functions either, as long as using such closures/state structures is easy and straightforward.

sizeof, typeof, compatible_types, and typenameof operators should be provided even by statically types languages.
C provides only the first one; GCC, Clang, Intel CC and others provide the two next ones as extensions to C; and the fourth one can only be approximated by preprocessor stringification.  This leads to less than optimal interfaces when values of runtime-dependent types need to be passed.  Typically, the calls need to explicitly specify the type as a literal (token or string in C) and the size of the value, but there is no good way to check they match the expression being passed.  (If the expression type is a structure with a C99 flexible array member, even sizeof won't work.)
In particular, stringifying any typeof (expression) will only lead to a string containing that text, not the type name.

These kinds of use cases are why I believe so strongly that looking at existing code (in different competing languages, suitable for the target niche) to understand what kind of approaches they use for solving these kinds of problems, is extremely important.  Because there is a lot of bad code out there, looking for consensus is not a good idea: you need to find the 'best' ways of solving each kind of a problem, and learn from that.  (To me, 'best' here means robust with errors easy to catch, without imposing a strict usage pattern; and not 'me like most'.)

These are the kinds of ideas I'm interested in, exactly the kinds if things that could be discussed and considered in the context of a new programming language. C doesn't have any concept of closures or anonymous functions (unless there are some vendor's who added these?).

As for nested procedures, first and foremost these are easy to incorporate into a new language, low cost (not free but hardly expensive). Code leveraging them is clear, easy to reason about. The state data the nested procedure can access is clear just by looking at the code. The inaccessibility of the nested function to other code is clear too. We can already nest scopes in C too, not a nested function but it is a nested scope. 

GCC has extensions that deliver nested function, someone did invest the effort to define and implement this.

Nested functions are not the same as closures either, they are different, perhaps one could achieve some desired outcome using either closures or nested functions but they are distinct concepts, not interchangeable.

Stuff like "typeof" and "typenameof" and so on, these too are things a new language could support. As a list of desirable features begins to grow, then so too does an encompassing paradigm, the language will begin to take shape by systematizing the way the features are defined.

Note too the plethora of C "extensions" by umpteen vendors, often incompatible, their presence tells us something important...




Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 29, 2022, 04:12:59 pm
If you look at the Pascal syntax diagram you will start to see where recursive descent was a part of the syntax and that nested procedures are the way to go:

I don't see the link.

Mutually-recursive procedures, yes. But not nested.

Try this:
Code: [Select]
procedure expression;
  var lattr: attr; lop: operator; typind: char; lsize: addrrange;

  procedure simpleexpression(fsys: setofsys);
    var lattr: attr; lop: operator; signed: boolean;

    procedure term(fsys: setofsys);
      var lattr: attr; lop: operator;

      procedure factor(fsys: setofsys);
var lcp: ctp; lvp: csp; varpart: boolean;
    cstpart: setty; lsp: stp;
      begin
if not (sy in facbegsys) then
  begin error(58); skip(fsys + facbegsys);
    gattr.typtr := nil

Page 138 and 139 of the syntax diagrams show the relationship between expression, simpleexpression, term and factor.  Note that factor can recursively call expression and start the entire tree over again.

The compiler code flows directly from the syntax diagrams.

Compared to the transition matrix we were taught in grad school, Wirth's approach looks elegant.

Wirth defined a simple language named PL/0 too that he used to convey this idea. I'll tell you another truly superb book on compiler design, one I used a lot in the past it is Understanding and Writing Compilers (https://www.amazon.com/Understanding-Writing-Compilers-do-yourself/dp/0333217322) by Richard Bornat. It is oriented to recursive descent and is very readable, the author has a deep understanding of machine architectures and code generation practicalities, gives easy to follow examples akin to the stuff Wirth wrote, I doubt I could have gotten as far as I did with my PL/I implementation without this book.

I have numerous books and back when I started PL/I there was no internet and books were the bed rock of technical work like this, The "Dragon Book" has its place of course and a few others but Bornat's book is right up there with the best of them, a much overlooked book IMHO.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 29, 2022, 04:48:47 pm
Now regarding coroutines, the following has emerged over the past week as I explore these. The original definition of couroutine (and the one embodied in the pseudo code below) is the one found in the 1963 paper, multiple execution contexts that can suspend and resume through mutual interaction.

In the emerging grammar, "coroutine" makes sense as an optional attribute one can place on a procedure definition, consider:

Code: [Select]
proc main
{

// left_offset = 0;
// left_frame = XXXX
// right_offset = 0;
// right_frame = YYYY
// curr_frame = left_frame;
// goto left_offset

call left (X); // must create a collective stack frame that all coroutines will (in essence) share.

}

proc left(L) coroutine
{

arg L;

// dcl locals

// do something

yield to right(Z); // actually a goto right_offset into right and set left_offset to 1 and curr_frame to right_frame

// do something else

yield to right(Z); // actually a goto right_offset into right and set left_offset to 2 and curr_frame to right_frame

}

proc right(R) coroutine
{

arg R;

// dcl locals

// do something

yield to left(W); // actually a goto left_offset into left and set right_offset to 1 and curr_frame to left_frame

// do something else

yield to left(W); // actually a goto left_offset into left and set right_offset to 2 and curr_frame to left_frame

}


The comments are simple, informal representations of how one might implement this, the register mechanics that arise from this behavior.  The compiler - by static analysis (i.e. by the "coroutine" attribute) - can see that a singe aggerate stack frame must be created when main initially invokes left via a conventional call. That results in a single stack frame that has working space - subframes - for both left and right, that stack frame exists until the call in main returns, this is conventional call/return.

The proc left executes normally until it reaches yield to. At that point a jump/goto starts execution at the address implied by right_offset (which lives in the aggregate stack frame) after first adjusting left_offset which represents where the execution of left will resume when right eventually does a yield to left.

Execution thus cycles back and forth, each procedure can access its args and locals because it has its own "slot" within the aggregate stack frame. Inside the procedures things look conventional, the code behaves conventionally.

The yield to can exist in the following variants, supporting coroutines and cofunctions:

Code: [Select]

yield (item);

yield to some_procedure;

yield to some_procedure(args);

yield (item) to some_procedure;

yield (item) to some_procedure(args);


The first is how a basic iterator could be written. Corresponding to these we have:

Code: [Select]

x = yield to some_procedure;

x = yield to some_procedure(args);

x = yield (y) to some_procedure(args);


where in its most general form:

Code: [Select]

a = yield (x) to somewhere(y);


Where a is the "returned" value that appeared in some other yield (value) clause in some other coroutine. The (y) arg also, becomes the current value of the argument defined the target coroutine's parameter list.

and so on.

So arguments can be passed into a coroutine and values can be returned from them, in a variety of ways. There is no setup/teardown of stack frames as the pair of coroutines executes, there is no pushing and popping of arguments either. The subframes exist and remain at all times, only when control eventually reaches the end of the final procedure (or a return is executed) does the aggregate stack frame vanish in the same way any stack frame vanishes when a called function/procedure terminates.

So this is an outline of the standard, historic coroutine pattern, it does what its described in the historic literature, and of course we can have more than two procedures/functions and we can even pass "function pointers" (to use C parlance) into them so that the code they "yield to" is defined at runtime not compile time as in my examples.

I've never really been able to find a clear detailed description of the classic historic coroutine, so some of this is a result of my efforts to make sense of the idea.

Thoughts?









Title: Re: A microcontroller programming language
Post by: rstofer on November 29, 2022, 05:01:07 pm

Wirth defined a simple language named PL/0 too that he used to convey this idea.
I have coded up that PL/0 compiler several times, once in Fortran.  Dozens of pages of Fortran or a very few pages of Pascal.  I'll concede that the Fortran was crap code but, really, sets aren't part of the language and they are terribly important to the compiler.

Fortunately, the CDC 6400 on which the compiler was developed had a 60 bit word and that was enough bits to accommodate the elements.  It gets ugly when you have to compare across 8 or 16 bit or even 32 bit entities.  Then there is the lack of formal recursion in Fortran (I believe I was using a IV version of Fortran).

Quote
I'll tell you another truly superb book on compiler design, one I used a lot in the past it is Understanding and Writing Compilers (https://www.amazon.com/Understanding-Writing-Compilers-do-yourself/dp/0333217322) by Richard Bornat. It is oriented to recursive descent and is very readable, the author has a deep understanding of machine architectures and code generation practicalities, gives easy to follow examples akin to the stuff Wirth wrote, I doubt I could have gotten as far as I did with my PL/I implementation without this book.
I ordered a copy from alibris.com - $15 including shipping.  I get a lot of used books from Alibris.  I suspect that's the way my library will go as well.
Title: Re: A new, hardware "oriented" programming language
Post by: eutectique on November 29, 2022, 05:05:17 pm
The only thing that annoys me is that the linker does not yet know how to sort those entries at link time.

There is SORT_BY_NAME (or simply SORT) keyword for input section description, does it not work?
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 29, 2022, 05:24:35 pm

Wirth defined a simple language named PL/0 too that he used to convey this idea.
I have coded up that PL/0 compiler several times, once in Fortran.  Dozens of pages of Fortran or a very few pages of Pascal.  I'll concede that the Fortran was crap code but, really, sets aren't part of the language and they are terribly important to the compiler.

Fortunately, the CDC 6400 on which the compiler was developed had a 60 bit word and that was enough bits to accommodate the elements.  It gets ugly when you have to compare across 8 or 16 bit or even 32 bit entities.  Then there is the lack of formal recursion in Fortran (I believe I was using a IV version of Fortran).

Quote
I'll tell you another truly superb book on compiler design, one I used a lot in the past it is Understanding and Writing Compilers (https://www.amazon.com/Understanding-Writing-Compilers-do-yourself/dp/0333217322) by Richard Bornat. It is oriented to recursive descent and is very readable, the author has a deep understanding of machine architectures and code generation practicalities, gives easy to follow examples akin to the stuff Wirth wrote, I doubt I could have gotten as far as I did with my PL/I implementation without this book.
I ordered a copy from alibris.com - $15 including shipping.  I get a lot of used books from Alibris.  I suspect that's the way my library will go as well.

Oh, well let me know what you think, I used the book in the early 90s then lost it, but bought a used copy from Amazon last year!

Title: Re: A microcontroller programming language
Post by: brucehoult on November 29, 2022, 06:35:13 pm
If you look at the Pascal syntax diagram you will start to see where recursive descent was a part of the syntax and that nested procedures are the way to go:

I don't see the link.

Mutually-recursive procedures, yes. But not nested.

Try this:
Code: [Select]
procedure expression;
  var lattr: attr; lop: operator; typind: char; lsize: addrrange;

  procedure simpleexpression(fsys: setofsys);
    var lattr: attr; lop: operator; signed: boolean;

    procedure term(fsys: setofsys);
      var lattr: attr; lop: operator;

      procedure factor(fsys: setofsys);
var lcp: ctp; lvp: csp; varpart: boolean;
    cstpart: setty; lsp: stp;
      begin
if not (sy in facbegsys) then
  begin error(58); skip(fsys + facbegsys);
    gattr.typtr := nil

Page 138 and 139 of the syntax diagrams show the relationship between expression, simpleexpression, term and factor.  Note that factor can recursively call expression and start the entire tree over again.

The compiler code flows directly from the syntax diagrams.

Compared to the transition matrix we were taught in grad school, Wirth's approach looks elegant.

There are three lines of actual executable code there, referring to the following names:

Code: [Select]
sy
facbegsys
error
skip
fsys
gattr

Of those, fsys is a formal argument of the current function (factor). All the others are undefined by the code shown here. "Free variables" if you will. Globals.

NONE of them are defined by any of the enclosing functions: expression, simpleexpression, term.

I can't see any necessity here for them to be nested functions instead of peer functions. It appears no use is being made of the nested scopes.
Title: Re: A microcontroller programming language
Post by: brucehoult on November 29, 2022, 06:57:03 pm
I have coded up that PL/0 compiler several times

I first did that in 1981, on an Apple ][, first in Pascal and then in machine code.

Quote
Fortunately, the CDC 6400 on which the compiler was developed had a 60 bit word and that was enough bits to accommodate the elements.  It gets ugly when you have to compare across 8 or 16 bit or even 32 bit entities.

It's really not a problem.

Set union and intersection are simply OR and AND iterated across the bytes or words of the set. Probably unrolled for any reasonable set, especially as you know at compile time the number of possible elements.

"elt IN set" is simply "(set[elt / WORDSIZE] & (1 << (elt % WORDSIZE))) != 0" where, if WORDSIZE is a power of 2 as is normal today, the / and % are themselves just a shift and an AND. Multi-word sets certainly would be more expensive on oddball word sizes such as 60 on the CDC or 12 on the PDP-8, but they are absolutely fine on 8, 16, 32, 64 bit machines. The worst is if you don't have constant-time single-instruction shifts. Shifting the 1 is the only one with a dynamic shift count, but it's also the simplest, so you could do it using an array of WORDSIZE elements if needed.
Title: Re: A microcontroller programming language
Post by: rstofer on November 29, 2022, 07:26:13 pm
I didn't want to paste a thousand lines of code so I only copied the first few lines just to show the hierarchy.  The remaining code is easy to find at the link.

Certainly the code could have been written without nesting but it is cleaner with it.  Only procedure 'term' can call procedure 'factor' so the nesting cleans up the namespace by default.  Going uphill, 'factor' can call 'expression' just not the other way around.  It's hard to provide this functionality with all procedures at the same level.

Title: Re: A microcontroller programming language
Post by: Mechatrommer on November 29, 2022, 07:34:34 pm
Quote
I'll tell you another truly superb book on compiler design, one I used a lot in the past it is Understanding and Writing Compilers (https://www.amazon.com/Understanding-Writing-Compilers-do-yourself/dp/0333217322) by Richard Bornat. It is oriented to recursive descent and is very readable, the author has a deep understanding of machine architectures and code generation practicalities, gives easy to follow examples akin to the stuff Wirth wrote, I doubt I could have gotten as far as I did with my PL/I implementation without this book.
I ordered a copy from alibris.com - $15 including shipping.  I get a lot of used books from Alibris.  I suspect that's the way my library will go as well.
learning on how to make an AI/physics/math/fancy data structure/class library in C or world/3d/game engine is much more fun and gives much more practical use... imho. cheers.
Title: Re: A microcontroller programming language
Post by: brucehoult on November 29, 2022, 08:31:29 pm
I didn't want to paste a thousand lines of code so I only copied the first few lines just to show the hierarchy.  The remaining code is easy to find at the link.

Certainly the code could have been written without nesting but it is cleaner with it.  Only procedure 'term' can call procedure 'factor' so the nesting cleans up the namespace by default.  Going uphill, 'factor' can call 'expression' just not the other way around.  It's hard to provide this functionality with all procedures at the same level.

But with this arrangement 'factor' can also call 'term', which is not permitted by the grammar. So this "functionality" is not actually doing anything much.

You'd be no worse off with top-level functions with 'expression' forward-declared, followed by 'factor', then 'term', then 'expression'.

Trying to enforce grammar rules by nested visibility in this way is not a useful thing to do, firstly because it is only partial, and secondly because the language grammar can be changed at the stroke of a pen ("requirements"), for example by adding a construct that wants to use 'term' or 'factor' somewhere else in the grammar that can't be a full 'expression'.
Title: Re: A microcontroller programming language
Post by: rstofer on November 29, 2022, 08:37:13 pm
learning on how to make an AI/physics/math/fancy data structure/class library in C or world/3d/game engine is much more fun and gives much more practical use... imho. cheers.

I have no artistic capability whatsoever.  Creating assets, particularly characters is out of the question.

I don't play video games - ever; I maxed out at PacMan.  I did look at Blender and Unreal just to get some idea of the workflow but I came to the immediate realization that it takes more than one person to create a game.  A cast of hundreds wouldn't surprise me.
Title: Re: A microcontroller programming language
Post by: brucehoult on November 29, 2022, 09:55:31 pm
learning on how to make an AI/physics/math/fancy data structure/class library in C or world/3d/game engine is much more fun and gives much more practical use... imho. cheers.

I have no artistic capability whatsoever.  Creating assets, particularly characters is out of the question.

I don't play video games - ever; I maxed out at PacMan.  I did look at Blender and Unreal just to get some idea of the workflow but I came to the immediate realization that it takes more than one person to create a game.  A cast of hundreds wouldn't surprise me.

You certainly need a variety of skills, which are not often found in one person.

Back in 2004 I was working for NZ Metservice, working on their TV weather graphics product, helping with modifications for a big new customer: the BBC in the UK.

They had hired a landscape painter, David Frampton, to improve the graphics elements. Which he did. He also gradually picked up a little bit of programming, which he hadn't previously done. After a while he decided to have a go at writing a simple game at home, and decided to start by doing a Mac reimplementation of the ancient "Choplifter" game that was on the Apple ][ of his youth. And C64, DOS, ...

In time he got it working, and started distributing it as shareware, called simply "Chopper", with very modest sales.

A couple of years later the iPhone was announced, and then a SDK for it, and an app store to be opened about six months later.

David decided to try porting Chopper to the iPhone. It worked. He got it in the app store on Day #1, in July 2008, one of very few apps available at the time. It was a pretty big hit. Enough that David paid off his mortgage and quit his job to work full-time on a new game.

Two years later, in July 2010, he released a full 3D update, "Chopper 2". It was an even bigger hit. David sold his house in Wellington and bought a lifestyle block near Napier and started working on another game.

Two and a half years later, in January 2013, he released "The Blockheads", something in the Minecraft genre, but a 2D side-scroller. Another huge hit. He spent another couple of years improving Blockheads before starting on a new game.

After seven years of development, in July 2022, David released "Sapiens". Review: "Could it be the best game of 2022?"  https://www.youtube.com/watch?v=ClkkDJA_15s (https://www.youtube.com/watch?v=ClkkDJA_15s)

All with David as the sole developer. The concept. The gameplay. The programming. The AI. The graphics. The music. And all self-funded, starting part time after hours with a salaried job.

David does employ someone to deal with support and forums. Just one person, I think.
Title: Re: A microcontroller programming language
Post by: Mechatrommer on November 29, 2022, 10:23:00 pm
I have no artistic capability whatsoever.
creating compiler that works with your machine, or better than the others, is an art in itself...
Title: Re: A microcontroller programming language
Post by: rstofer on November 29, 2022, 11:07:41 pm
I have no artistic capability whatsoever.
creating compiler that works with your machine, or better than the others, is an art in itself...

And best left to professionals like Niklaus Wirth and the other luminaries of computer science.

Not that it matters but the IBM1130 Fortran compiler was a 27 pass beast.  It had to run in as little as 4k words on a diskless paper tape machine.  The card deck (I'm not sure the source could be on paper tape...) was read into RAM and shared space with the compiler phases.  As the source was digested, the text was replaced with code.

It has a little more breathing room on my 32k word FPGA incantation.  Having a disk helps.  I didn't bother implementing paper tape.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 29, 2022, 11:18:28 pm
When defining a new language, at the stage where irreversible decisions are yet to be made, there are a number of things we can look at in a fresh light.

One of these is the lexical structure of numeric literals, a tiny change to the token definitions for these means we can easily support optional commas:

Code: [Select]

   rate = 128,000;

   freq = 100,000,000;

   ramp = 12,456.558;


None of these present any problems at all insofar as lexical analysis ("recognizing") is concerned and have a definite pragmatic appeal. Of course there are cultural differences, but since we're not dealing with data but rather programming language notation, the comma is no less reasonable than the period.



Title: Re: A microcontroller programming language
Post by: coppice on November 30, 2022, 01:26:13 am
When defining a new language, at the stage where irreversible decisions are yet to be made, there are a number of things we can look at in a fresh light.

One of these is the lexical structure of numeric literals, a tiny change to the token definitions for these means we can easily support optional commas:

Code: [Select]

   rate = 128,000;

   freq = 100,000,000;

   ramp = 12,456.558;


None of these present any problems at all insofar as lexical analysis ("recognizing") is concerned and have a definite pragmatic appeal. Of course there are cultural differences, but since we're not dealing with data but rather programming language notation, the comma is no less reasonable than the period.
Commas inserted like that are very confusing in countries which don't use that notation, and especially in countries that use a comma as the decimal point. Software exists in a very international market. Try to keep it as generic as possible. Should we support numbers like 1,00,00,000? Indians would be very happy with that notation. I probably wouldn't.
Title: Re: A microcontroller programming language
Post by: bpiphany on November 30, 2022, 07:43:56 am
Blatantly stolen, from here https://stackoverflow.com/questions/54009778/what-do-underscores-in-a-number-mean

Code: [Select]
a = 1_00_00  # you do not need to group digits by 3!
b = 0xbad_c0ffee  # you can make fun with hex digit notation
c = 0b0101_01010101010_0100  # works with binary notation
f = 1_000_00.0
print(a,b,c,f)

It's not pretty, and I prefer to specify big number in other fashions myself. But if you have an integer/constant/bitmask with tons of significant digits I suppose it may be helpful.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on November 30, 2022, 09:09:41 am
Where are [compatible_types and typenameof] useful in C?
typenameof (expression) would correspond to C STRINGIFY(typeof(expression)) if it was supported and actually worked.  Essentially, it allows a library to implement run-time interfaces like
    #define  register_type(expr)  lib_register_type(typenameof (expr), sizeof (expr))
    mytype_id = register_type(type-or-expression);
for registering a type dynamically, for example in a message passing or event library, and
    #define  send(handle, obj)  lib_send((handle), &(obj), sizeof (obj), typenameof (obj))
    result = send(handle, object);
to send not just the storage representation of an object, but also its type name and size, so the recipient knows what it is receiving.

Currently, the two interfaces have to be implemented as something like
    mytype_id = register_type("typename", size);
    result = send(handle, pointer, size, "typename");
which are prone to bugs (especially bitrot), because there is no way to check that typename actually matches the type, for example when due to a bugfix or similar the original type is modified, but not all places where its type is mentioned are correctly updated to reflect the change.

compatible_types matches the GCC/Clang/Intel CC __builtin_compatible_types_p() C extension.  If you support macros, it allows you to do type inference with a compile-time fixed set of types, for example in C,
    #define  TYPECODE(type)  (__builtin_compatible_types_p(type, uint8_t)  ? TYPE_U8  : \
                              __builtin_compatible_types_p(type, int8_t)   ? TYPE_I8  : \
                              __builtin_compatible_types_p(type, uint16_t) ? TYPE_U16 : \
                              __builtin_compatible_types_p(type, int16_t)  ? TYPE_I16 : \
                              __builtin_compatible_types_p(type, uint32_t) ? TYPE_U32 : \
                              __builtin_compatible_types_p(type, int32_t)  ? TYPE_I32 : \
                              __builtin_compatible_types_p(type, uint64_t) ? TYPE_U64 : \
                              __builtin_compatible_types_p(type, int64_t)  ? TYPE_I64 : TYPE_UNKNOWN)
Unlike comparing typenames, on architectures where char, short, int, long, and long long match the above specific-size integer types (they likely shall on all in C23), they too will match to the known types using the above macro.
With this, the type of an expression can be detected and specified at runtime, for example when a compile-time fixed set of types are supported by a library, but the user of that library wants to use their own type names (say, u16 and such).

Note that !strcmp(typenameof (a), typenameof (b)) is not the same as __builtin_compatible_types_p(typeof(a), typeof(b)).  If we consider typedef a b;, the former yields false (0), whereas the latter yields true (1).

Most important use cases for both are about not having to specify the type of an object or an expression, when already supplying that object or expression, because re-stating the type is prone to bugs.  One might think that these are corner cases, but fact is, such bugs are unfortunately common, and could be completely avoided with these two compile-time keywords.

The only thing that annoys me is that the linker does not yet know how to sort those entries at link time.
There is SORT_BY_NAME (or simply SORT) keyword for input section description, does it not work?
I was referring there to sorting the contents of a section, treating it as an array of N structures of size L with a sort key of type T at offset K.

I guess one could use custom dotted name for each entry (.sectionname.key), but it'd only work for keys that are valid section names.
SORT_BY_INIT_PRIORITY works for .sectionname.nonneg section names, where nonneg is a nonnegative integer between 0  and 65535, inclusive.
(I haven't even found a way yet to generate unique section name suffixes that work (are unique) across source and object files.)

The reason it would be nice for the linker to be able to do this for a small number of key types, is to keep the darn relocation entries in sync with the contents.
I know, I should just stop whining, and go do it portably using bfd (part of binutils) or libelf (non-binutils ELF library).
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 30, 2022, 03:19:08 pm
When defining a new language, at the stage where irreversible decisions are yet to be made, there are a number of things we can look at in a fresh light.

One of these is the lexical structure of numeric literals, a tiny change to the token definitions for these means we can easily support optional commas:

Code: [Select]

   rate = 128,000;

   freq = 100,000,000;

   ramp = 12,456.558;


None of these present any problems at all insofar as lexical analysis ("recognizing") is concerned and have a definite pragmatic appeal. Of course there are cultural differences, but since we're not dealing with data but rather programming language notation, the comma is no less reasonable than the period.
Commas inserted like that are very confusing in countries which don't use that notation, and especially in countries that use a comma as the decimal point. Software exists in a very international market. Try to keep it as generic as possible. Should we support numbers like 1,00,00,000? Indians would be very happy with that notation. I probably wouldn't.

Thanks for the remarks. If you do consider the cultural differences then clearly we should not restrict the decimal point to being a PERIOD yet we do, universally in all programming languages. If we are prepared to restrict the decimal point to a PERIOD then we should also - if we're being consistent - be equally prepared to accept the COMMA as the thousands separator surely? Remember too this will be optional, there'll be no compulsion to use a comma at all, only the ability for the language to tolerate if it is present.

Given that programming languages (almost) universally reflect the English speaking, Western culture (keywords are all English for example) extending that to the optional inclusion of the COMMA seems entirely reasonable to me.

However there are standards/recommendations for this and the general feeling seems to be that only SPACE should be allowed as the thousands/grouping separator.

So perhaps we could do something else, something more flexible, examples:

123,456.87
123 456.87
1,23,456.87
1 23 456.87
123 456,87

The rules for this could simply be that the decimal separator be defaulted to PERIOD with the option to set it to COMMA. Then any separators be entirely optional and allowed to be SPACE COMMA PERIOD UNDERSCORE or APOSTROPHE so long as it is not the same as the defined separator.

Any of these legal separators have no influence on the numbers actual magnitude, they are purely a visual aid so will ultimately be simply removed when the token is created. This might all look rather long- winded but we either disregard cultural variations and impose Western English rules (12,345,678.965) or we accommodate cultural variants to some reasonable degree. Since the separator carries no weighting or anything (it just a visual aid) so in other words allow not only _ as some languages do but let that also be SPACE, COMMA etc if desired, perhaps insisting too that a separator must be alone, one cannot have two commans or two underscores etc.

















Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 30, 2022, 03:33:52 pm
Blatantly stolen, from here https://stackoverflow.com/questions/54009778/what-do-underscores-in-a-number-mean

Code: [Select]
a = 1_00_00  # you do not need to group digits by 3!
b = 0xbad_c0ffee  # you can make fun with hex digit notation
c = 0b0101_01010101010_0100  # works with binary notation
f = 1_000_00.0
print(a,b,c,f)

It's not pretty, and I prefer to specify big number in other fashions myself. But if you have an integer/constant/bitmask with tons of significant digits I suppose it may be helpful.

Yes, that language (like C#) tolerates any number of UNDERSCORE chars placed at any position. The base indicator can though be a trailing character rather than leading. I'm also finding that another separator for that base character also makes the text visually easier om the eye, consider then (and combining this with the earlier posts about "thousands" and so on):

Code: [Select]

a = 10:H; // 16 in hex
b = 1245 77AC:H; // also hex with optional separator
c = 1001 0110:B;  // binary with space
d = 123,456.77;   //  conventional decimal with separator
e = 123,456.77:D // explicit base - optional.


I'm also inclined to insist that hexadecimal must be uppercase too for consistency and that the base char also be uppercase.

It looks like all of these can be accommodated without any problems (in a new language that is) but if we do allow COMMA as a separator we would need to remove it as the separator for arguments:

Code: [Select]

call initialize(100,345.88,severity);


That is ambiguous now, it could be 100 and 345.88 and severity or it could be 100,345.88 and severity, that is two args or three args...



Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 30, 2022, 04:37:33 pm
I'm of the opinion that the comma is too ubiquitous to abandon as a separator in things like argument lists. So the separator for large numbers should be either single spaces or single underscores, and the decimal point is always the period. The position of the separator(s) is arbitrary.

Hexadecimal should be flexible case too, though mixing case should be an error, the base specifier as COLON followed by a single letter seems fine, and that can be upper or lower case.

This strikes me as reasonable...
Title: Re: A microcontroller programming language
Post by: coppice on November 30, 2022, 06:15:58 pm
If you do consider the cultural differences then clearly we should not restrict the decimal point to being a PERIOD yet we do, universally in all programming languages.
That's not quite true. Have you ever seen French or German Fortran? There was a lot of localisation of computer languages in the early days. The keywords were localised. Things like decimal points and other formatting issues were localised. However, it made such a pain of passing stuff around in multi-national organisations that by the end of the 60s computer languages has pretty much abandoned this approach.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 30, 2022, 07:57:19 pm
If you do consider the cultural differences then clearly we should not restrict the decimal point to being a PERIOD yet we do, universally in all programming languages.
That's not quite true. Have you ever seen French or German Fortran? There was a lot of localisation of computer languages in the early days. The keywords were localised. Things like decimal points and other formatting issues were localised. However, it made such a pain of passing stuff around in multi-national organisations that by the end of the 60s computer languages has pretty much abandoned this approach.

You are absolutely right, thanks, I stand corrected, Algol 68 too it seems.

I was reading too of this rather ambitious project Citrine (https://en.wikipedia.org/wiki/Non-English-based_programming_languages#Citrine).

It would be very easy indeed to support alternatively spelled keywords (especially here where the grammar has no reserved words) but I'd consider that not a "phase 1" thing for me!

So far as the numeric notation goes, I've settled for this, these are all examples of stuff that the lexer is recognizing just now:

Code: [Select]

FA068AB7:H // hex literal
FA06 8AB7:H // hex literal
FA06_8AB7:H // hex literal
FA06   __ 83BC  8AB7:H // hex literal (I'm going to reject such a monster though either have space OR underscore and no more than one at a time)

FA068AB7 // ordinary identifier. An identifier composed wholly of hex characters should perhaps be disallowed, inadvisable...

FA06 8AB7  :  H // hex literal


Admittedly this isn't a huge chunk of functionality but considering that MCU developers regularly have to code decimal, hex and binary literals, this kind of flexibility in how to express them can only be a good thing.








Title: Re: A microcontroller programming language
Post by: rstofer on November 30, 2022, 08:57:12 pm
It would be very easy indeed to support alternatively spelled keywords (especially here where the grammar has no reserved words) but I'd consider that not a "phase 1" thing for me!
There was at least one Fortran compiler where DAMNITALL could be used for DIMENSION - I guess the counted 9 chars starting with D and called it good.  They could almost get away with that as long as all declarations had to precede any executable statements.  Maybe they looked for the 3rd char to be M, I don't know...
Quote
So far as the numeric notation goes, I've settled for this, these are all examples of stuff that the lexer is recognizing just now:

Code: [Select]

FA068AB7 // ordinary identifier. An identifier composed wholly of hex characters should perhaps be disallowed, inadvisable...


Including FACE?  Seems a reasonable variable name for image recognition.

I can't say I'm a fan of the proposal.  I find the C notation more than adequate for common bases and I'm not too sure there is any utility for other bases.  Even base 8 is going the way of Digital Equipment Corporation - remember them?  Realistically, for coding we have bases 2, 8 and 16. We run out of English language letters before we can do base 32 (fortunately).

I'm also not a fan of the parser having to look so far ahead to determine whether there is a numeric constant or just another variable. Having numerics start with a decimal digit seems reasonable.

One thing about Pascal syntax, the compiler knew exactly what set of symbols could come next at every step of parsing.  No backtracking that I know of.

I do like being able to use underscores with long numeric strings but that's as far as I would push it.  OTOH, I'm not creating the language...
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on November 30, 2022, 09:20:22 pm
It would be very easy indeed to support alternatively spelled keywords (especially here where the grammar has no reserved words) but I'd consider that not a "phase 1" thing for me!
There was at least one Fortran compiler where DAMNITALL could be used for DIMENSION - I guess the counted 9 chars starting with D and called it good.  They could almost get away with that as long as all declarations had to precede any executable statements.  Maybe they looked for the 3rd char to be M, I don't know...
Quote
So far as the numeric notation goes, I've settled for this, these are all examples of stuff that the lexer is recognizing just now:

Code: [Select]

FA068AB7 // ordinary identifier. An identifier composed wholly of hex characters should perhaps be disallowed, inadvisable...


Including FACE?  Seems a reasonable variable name for image recognition.

I can't say I'm a fan of the proposal.  I find the C notation more than adequate for common bases and I'm not too sure there is any utility for other bases.  Even base 8 is going the way of Digital Equipment Corporation - remember them?  Realistically, for coding we have bases 2, 8 and 16. We run out of English language letters before we can do base 32 (fortunately).

I'm also not a fan of the parser having to look so far ahead to determine whether there is a numeric constant or just another variable. Having numerics start with a decimal digit seems reasonable.

One thing about Pascal syntax, the compiler knew exactly what set of symbols could come next at every step of parsing.  No backtracking that I know of.

I do like being able to use underscores with long numeric strings but that's as far as I would push it.  OTOH, I'm not creating the language...

You're right FACE is entirely reasonable!

There's no parsing here at this stage, all of these tokens can be recognized with an FSM based lexical analyzer (no stack involved), this stuff gets done well before the parser sees anything.

Speaking personally, I've always found it a contrivance to use underscores as a numeric separator (as in C#), as soon as one does that then the thing looks like an identifier (to me) even though it isn't. Then we have the leading 0x or 0b to designate the base, I mean leading? why must there be a "0" at all? and what does "x" have to do with 16? Many of these practices are due to the fact that C's grammar and lexical structure impose unhelpful constraints, this is a major reason I am critical of it's influence.

However, it would be sensible to accommodate that practice, C is ubiquitous so both the C "way" of representing these and the alternative are easily supported.

Here's some chat (https://stackoverflow.com/questions/27767781/why-was-the-space-character-not-chosen-for-c14-digit-separators) about this.
Title: Re: A microcontroller programming language
Post by: rstofer on November 30, 2022, 10:06:38 pm
Another use for the underscore at less uniform locations is when we want to separate fields in a value being sent to a control register.  Some fields are 1 bit, some are 3 bits, and so on.  Separating the fields could be quite handy.
Title: Re: A microcontroller programming language
Post by: newbrain on November 30, 2022, 10:53:58 pm
Just to make sure some wheels do not get reinvented too much, C23 provides:
* typeof (https://www.open-std.org/JTC1/SC22/WG14/www/docs/n2912.pdf#subsubsection.6.7.2.5)
* bit-precise integer types (https://www.open-std.org/JTC1/SC22/WG14/www/docs/n2912.pdf#subsection.6.2.5)
* binary literals, and a digit separator (https://www.open-std.org/JTC1/SC22/WG14/www/docs/n2912.pdf#subsubsection.6.4.4.1) (')
* (optional)decimal floating point types (https://www.open-std.org/JTC1/SC22/WG14/www/docs/n2912.pdf#subsection.6.2.5)
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 01, 2022, 09:30:15 am
Just to make sure some wheels do not get reinvented too much, C23 provides:
* typeof (https://www.open-std.org/JTC1/SC22/WG14/www/docs/n2912.pdf#subsubsection.6.7.2.5)
* bit-precise integer types (https://www.open-std.org/JTC1/SC22/WG14/www/docs/n2912.pdf#subsection.6.2.5)
* binary literals, and a digit separator (https://www.open-std.org/JTC1/SC22/WG14/www/docs/n2912.pdf#subsubsection.6.4.4.1) (')
* (optional)decimal floating point types (https://www.open-std.org/JTC1/SC22/WG14/www/docs/n2912.pdf#subsection.6.2.5)

This is quite helpful. Their analysis is interesting to study and it indicates the perceived utility of the concepts and ideas.

I don't see the purpose of typeof here though. In C# a Type is a class representing a type, metadata describing a type. The typeof in C# gets that type from a type's name. In a generic setting typeof(T) returns the actual runtime type represented by T.

In C every type is known at compile time, there are no 'objects' who's type we cannot know, so what's it for?
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 01, 2022, 10:45:35 am
Something a new language needs is a precisely defined means of initializing variables and structure instances. I've seen { 0 } can do this but only at the point of declaration it seems.

C# has a default() language function, so something like that makes a lot of sense. This would initialize every member field (recursively) to a value suitable for it's type.
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 01, 2022, 11:38:10 am
Something a new language needs is a precisely defined means of initializing variables and structure instances. I've seen { 0 } can do this but only at the point of declaration it seems.

Again, you assume the worst when you simply don't know.

But I can assure you, you can set a struct to {0} any time you want; this is not limited to the initialization at point of declaration.

What may confuse the young players is, because C evaluates right side of expression "on their own" (which simplifies rules and removes guesswork), the compiler can't know the type from {0} alone. In an initializer, it is known (from the variable you are initializing), but in assignment, you need to tell the type:

typedef struct
{
    int asdf;
    int qwerty;
} our_type_t;

our_type_t object = {0};
object = (our_type_t){0};
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 01, 2022, 01:07:25 pm
Something a new language needs is a precisely defined means of initializing variables and structure instances. I've seen { 0 } can do this but only at the point of declaration it seems.

Again, you assume the worst when you simply don't know.

But I can assure you, you can set a struct to {0} any time you want; this is not limited to the initialization at point of declaration.

What may confuse the young players is, because C evaluates right side of expression "on their own" (which simplifies rules and removes guesswork), the compiler can't know the type from {0} alone. In an initializer, it is known (from the variable you are initializing), but in assignment, you need to tell the type:

typedef struct
{
    int asdf;
    int qwerty;
} our_type_t;

our_type_t object = {0};
object = (our_type_t){0};

Thank you for the helpful (if slightly condescending) clarification.

The restriction is bizarre though, surely the compiler can easily infer that cast itself, but such is C. I suppose one could craft a macro for this initialization, one that uses typeof.
Title: Re: A microcontroller programming language
Post by: newbrain on December 01, 2022, 01:17:20 pm
easily infer that cast itself
That's not a cast.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 01, 2022, 01:23:24 pm
easily infer that cast itself
That's not a cast.

Very well but it is regularly referred to by that term, and it could be inferred, the compiler has access to enough information to make the inference.

Title: Re: A microcontroller programming language
Post by: Siwastaja on December 01, 2022, 03:50:07 pm
Very well but it is regularly referred to by that term, and it could be inferred, the compiler has access to enough information to make the inference.

Compiler could have that information, yes - but this is one of the fundamental rules of C: in assignments, right side is evaluated on its own, with no information fed-forward from the left side.

This again makes the language rules simpler and more predictable.

The same confusion arises when doing arithmetics with different types:
uint32_t a;
uint32_t b;
uint64_t c;

c = a + b;

Per language rules, the result of (a+b) will be of type uint32_t, and only during the assignment is converted to uint64_t. So if a and b are near the limit or numerical range, the result will wrap around; maybe the programmer's intent was:
c = (uint64_t)a + (uint64_t)b;

Maybe a more "clever" language, where the type of destination of assignment was considered, would be good? It would remove one footgun. On the other hand, it would create more subtle ones, as the rules of conversions would get even more complicated than they already are. With current solution, you at least quickly learn the rule - right side is in isolation - and can control the necessary conversions yourself.

In fact, we can go further - if you want such automation, why stop there? Why not remove all the manual type hassle completely? Python does that, and infers the type automagically for you. Some like it, some do not; but clearly the use cases are different. Those who work with microcontrollers, often need the control.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 01, 2022, 04:56:29 pm
Very well but it is regularly referred to by that term, and it could be inferred, the compiler has access to enough information to make the inference.

Compiler could have that information, yes - but this is one of the fundamental rules of C: in assignments, right side is evaluated on its own, with no information fed-forward from the left side.

This again makes the language rules simpler and more predictable.

The same confusion arises when doing arithmetics with different types:
uint32_t a;
uint32_t b;
uint64_t c;

c = a + b;

Per language rules, the result of (a+b) will be of type uint32_t, and only during the assignment is converted to uint64_t. So if a and b are near the limit or numerical range, the result will wrap around; maybe the programmer's intent was:
c = (uint64_t)a + (uint64_t)b;

Maybe a more "clever" language, where the type of destination of assignment was considered, would be good? It would remove one footgun. On the other hand, it would create more subtle ones, as the rules of conversions would get even more complicated than they already are. With current solution, you at least quickly learn the rule - right side is in isolation - and can control the necessary conversions yourself.

In fact, we can go further - if you want such automation, why stop there? Why not remove all the manual type hassle completely? Python does that, and infers the type automagically for you. Some like it, some do not; but clearly the use cases are different. Those who work with microcontrollers, often need the control.

This all reduces to how the language determines the type of an expression. I accept that, it's reasonable as is the arithmetic example you cite. The resultant type of expressions is indeed in most languages, deterministically inferred by the types of the operands and the nature of the operators - all good.

And yes some languages can implicitly convert the resultant expression's type to the target's type and some will not.

But here we are dealing with something else, I mean I can write this in C:

Code: [Select]

int ZZZ = 123456;
char MMM;

        MMM = ZZZ;


The compiler does not object and insist I do this:

Code: [Select]

int ZZZ = 123456;
char MMM;

        MMM = (char)ZZZ;


So it's rather idiosyncratic (there's that word again) of C to insist we code:

Code: [Select]

MyType datum;

datum = (MyType){0};


Unless I'm mistaken, this not a case of potential ambiguity, it seems that unless one does (MyType){0} then it will not compile, nothing else is permitted so it could only ever be interpreted that one way.

Look at every example in real code, you'll never find an example of:

Code: [Select]

SomeType A;

A = (SomeOtherType){0};


At least I don't imagine you ever will, I might be wrong though...






Title: Re: A microcontroller programming language
Post by: Siwastaja on December 01, 2022, 05:48:07 pm

Code: [Select]

int ZZZ = 123456;
char MMM;

        MMM = ZZZ;


The compiler does not object and insist I do this:

Because compiler knows ZZZ is of type int! Hence, ZZZ can be used as right-side operator in assigments, it can be used in comparisons, arithmetics - wherever.

But you can't write

datum = {0};

only because the language has no idea how to interpret {0}. At lowest level, how many bytes does {0} consist of? No idea, without the type. Compared to that, 0 is treated as int internally. But there is no such default interpretation for {0} (what would it be?), so you need to specify the type (with the cast-like syntax).

Quote
Look at every example in real code, you'll never find an example of:

Code: [Select]

SomeType A;

A = (SomeOtherType){0};


You won't find such example, because it won't compile, because that assignment is invalid as there is no implicit conversion available for the assignment. Yes, C is more strongly typed than you might think!

Code: [Select]
typedef struct { int a; } type1_t;
typedef struct { char b; } type2_t;

int main()
{
        type1_t t1;

        t1 = (type1_t){0}; // OK
        t1 = (type2_t){0}; // error

        return 0;
}

hrst@jorma:~$ gcc t2.c
t2.c: In function ‘main’:
t2.c:10:7: error: incompatible types when assigning to type ‘type1_t’ {aka ‘struct <anonymous>’} from type ‘type2_t’ {aka ‘struct <anonymous>’}
   10 |  t1 = (type2_t){0};


This is why you need to decide how to convert between those types, and create a conversion function, like:

Code: [Select]
twotypes.h

typedef struct {uint32_t thing} thing_wide_t;    // high numerical range thing
typedef struct {uint16_t thing} thing_narrow_t; // low-cost version of thing, same accuracy, less range

static inline thing_narrow_t wide_to_narrow(thing_wide_t in)
{
    thing_narrow_t ret;
    ret.thing = (in.thing > UINT16_MAX):UINT16_MAX:in.thing; // saturate instead of overflow
    return ret;
}


Built-in integer types, for example, have built-in conversions.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 01, 2022, 07:09:05 pm

Code: [Select]

int ZZZ = 123456;
char MMM;

        MMM = ZZZ;


The compiler does not object and insist I do this:

Because compiler knows ZZZ is of type int! Hence, ZZZ can be used as right-side operator in assigments, it can be used in comparisons, arithmetics - wherever.

But you can't write

datum = {0};

only because the language has no idea how to interpret {0}. At lowest level, how many bytes does {0} consist of? No idea, without the type. Compared to that, 0 is treated as int internally. But there is no such default interpretation for {0} (what would it be?), so you need to specify the type (with the cast-like syntax).

Quote
Look at every example in real code, you'll never find an example of:

Code: [Select]

SomeType A;

A = (SomeOtherType){0};


You won't find such example, because it won't compile, because that assignment is invalid as there is no implicit conversion available for the assignment. Yes, C is more strongly typed than you might think!

Code: [Select]
typedef struct { int a; } type1_t;
typedef struct { char b; } type2_t;

int main()
{
        type1_t t1;

        t1 = (type1_t){0}; // OK
        t1 = (type2_t){0}; // error

        return 0;
}

hrst@jorma:~$ gcc t2.c
t2.c: In function ‘main’:
t2.c:10:7: error: incompatible types when assigning to type ‘type1_t’ {aka ‘struct <anonymous>’} from type ‘type2_t’ {aka ‘struct <anonymous>’}
   10 |  t1 = (type2_t){0};


This is why you need to decide how to convert between those types, and create a conversion function, like:

Code: [Select]
twotypes.h

typedef struct {uint32_t thing} thing_wide_t;    // high numerical range thing
typedef struct {uint16_t thing} thing_narrow_t; // low-cost version of thing, same accuracy, less range

static inline thing_narrow_t wide_to_narrow(thing_wide_t in)
{
    thing_narrow_t ret;
    ret.thing = (in.thing > UINT16_MAX):UINT16_MAX:in.thing; // saturate instead of overflow
    return ret;
}


Built-in integer types, for example, have built-in conversions.


Yes it knows the type, I understand that. It can also unambiguously infer the type for (Type){0} because it can only ever be one type! this is a good example of a language weakness, if they changed the language to do that type inference (in this specific circumstance) nothing would or could break.

But what you said earlier appears to be incorrect, even in the declaration the RHS has an unknown type!

Code: [Select]

DataStructure datum = {0};


In that snippet, it must be examining the type of datum in order to know what {0} means! So there's no good reason why that same process can't be followed in the later, assignment case.

Code: [Select]

MyStructure a = {0};
MyStructure b;

b = {0};

As a compiler writer, I'm having a hard time understanding the difficulty, surely there's more to all this?

Anyway it is what it is, I'm just less forgiving than you, I have higher, yet reasonable expectations of a serious programming language.

Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 01, 2022, 08:45:14 pm
Code: [Select]

MyStructure a = {0};
MyStructure b;

b = {0};

As a compiler writer, I'm having a hard time understanding the difficulty, surely there's more to all this?

In C, a literal constructed with curly brackets has no definite type, and its type can't be derived from the literal itself. What is '{0}'? Are you able to give it a type? Nope.
What is '{1,2,3}'? Not any better. Could be an array (of what), could be a struct (with what members?)

For historical reasons, C accepts such *initializers* for practical purposes. In this context, this in an initializer, not a literal that has semantic by itself.

In C99, they introduced compound literals. But you have to give them a type, otherwise they have none. For backwards compatibility, C99 still accepts initializers which are not fully-formed (so, given a type) compound literals. But it doesn't in the context of an assignment, because in this context, the literal has no type and is thus not an expression you can use (again apart from the context of initialization.)

This is all a semantic question. Sure a compiler could 'guess' the type of a compound literal without explicitely giving it a type in the context of an assignment, but it wouldn't fit in the C language with its promotion and implicit conversion rules.

Fact is, defining a syntax with which 'complex' literals (so beyond base scalar types) fully embed their own type is hard. Depending on the "strength" of your type system, it may or may not matter. For a weakly-type language, or even a dynamically-typed language, it may not matter at all. Also depends on what kind of grammar you define and how context-dependent it is.

So can the above proposal you made be implemented? Of course. Are there "corner cases" for which it could be less trivial? Probably.
If you find that useful, why not. But I have enough gripes about all the implicit conversions in C which allow pretty sloppy programming, so I'm not sure I'd want to add a layer of that with "typeless" compound literals.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 01, 2022, 09:20:05 pm
Code: [Select]

MyStructure a = {0};
MyStructure b;

b = {0};

As a compiler writer, I'm having a hard time understanding the difficulty, surely there's more to all this?

In C, a literal constructed with curly brackets has no definite type, and its type can't be derived from the literal itself. What is '{0}'? Are you able to give it a type? Nope.
What is '{1,2,3}'? Not any better. Could be an array (of what), could be a struct (with what members?)

For historical reasons, C accepts such *initializers* for practical purposes. In this context, this in an initializer, not a literal that has semantic by itself.

In C99, they introduced compound literals. But you have to give them a type, otherwise they have none. For backwards compatibility, C99 still accepts initializers which are not fully-formed (so, given a type) compound literals. But it doesn't in the context of an assignment, because in this context, the literal has no type and is thus not an expression you can use (again apart from the context of initialization.)

This is all a semantic question. Sure a compiler could 'guess' the type of a compound literal without explicitely giving it a type in the context of an assignment, but it wouldn't fit in the C language with its promotion and implicit conversion rules.

Fact is, defining a syntax with which 'complex' literals (so beyond base scalar types) fully embed their own type is hard. Depending on the "strength" of your type system, it may or may not matter. For a weakly-type language, or even a dynamically-typed language, it may not matter at all. Also depends on what kind of grammar you define and how context-dependent it is.

So can the above proposal you made be implemented? Of course. Are there "corner cases" for which it could be less trivial? Probably.
If you find that useful, why not. But I have enough gripes about all the implicit conversions in C which allow pretty sloppy programming, so I'm not sure I'd want to add a layer of that with "typeless" compound literals.

I was simply pointing out in my last post the question you ask "What is '{0}'? Are you able to give it a type? Nope." applies in this case too surely, where the expression also appears alone on the RHS of an assignment:

Code: [Select]

MyStructure a = {0};


The compiler knows no more or no less in that case than it does here:

Code: [Select]

MyStructure a;

a = {0};


In both cases, whatever it is able to "know" about "a" must be exactly the same, the only way in can "know" the type in the first example is by looking at the type of "a". I'm not proposing it do that, it's too late, not gonna happen, the language is frankly beyond repair s far as I can see, there seems to be no good design reason for this, it look like laziness or perhaps fragility to existing compiler implementations, I really can't make sense of it.

The question came up due to a different issue and that is the absence of a simply means of initializing any such type, an operation that initializes every member to a suitable default for the type, zero for arithmetic types, null for pointers and so on.

I'd imagine that I'd implement that as something like:

Code: [Select]

dcl a MyStructure = default();

// code...

a = default();

// for any type.


Where default() is a builtin language function. Consider too that setting a pointer "to zero" is not really what the C standard says. It's expected (if my understanding is correct) that when the compiler sees some_pointer = 0; it will - under the hood - set the numeric value to some implementation/target defined value, almost always zero but not quite always. So again there should also be a null() language function or something.

In fact on the Stratus VOS operating system in the past, that did use the numeric value 1, I seem to recall the explanation (I heard on an OS internals class in the late 1980s) that there was some very good hardware basis for this. That system used multiple 68000 (family) processors actually. It was something to do with the fact that 0 is an even number and 1 is odd, and that was important for them.

Here we go: VOS C Language (https://stratadoc.stratus.com/vos/14.7.2/r040-02/wwhelp/wwhimpl/js/html/wwhelp.htm?context=r040-02&file=ch4r040-02i.html).
Title: Re: A microcontroller programming language
Post by: gf on December 02, 2022, 03:25:28 am
In c++ you can already do that, with slightly different syntax:

Code: [Select]
MyStructure a();
...
a = MyStructure();

And there is also nullptr:

Code: [Select]
some_pointer = nullptr;
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 02, 2022, 08:16:10 am
Initialization is indeed exception to the rule. Such is life, rules are arbitrary and not always perfectly logical. That does not mean rules are stupid or useless. I don't mind typing a bit of extra information, when it conveys important information for us programmers, like the type. This adds safety; the C standard requiring me to type the type (pun intended), means:

* I need to keep my brain enabled, less likely to do a mistake with the type
* If I make a mistake and use a wrong type, the compiler will check it for me and error out
* If I want to assign or compare incompatible types, that is only possible by writing the required conversion functions (or doing nasty pointer casting etc., which pokes in the eye)

This is a feature of a strongly typed language. And yet, I would prefer C to be even more strongly typed. Implicit conversions are classic footguns. I don't want any more of them, and oh boy how much I struggled with PHP and its dynamic typing system (although PHP is a disaster in every other regard, too).

Trying to make typing a bit faster is not a valid target. Less than 1% of time in programming is spent actually writing characters on code editor screen.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 02, 2022, 09:08:42 am
I don't mind typing a bit of extra information, when it conveys important information for us programmers, like the type. This adds safety; the C standard requiring me to type the type (pun intended), means:

* I need to keep my brain enabled, less likely to do a mistake with the type
* If I make a mistake and use a wrong type, the compiler will check it for me and error out
* If I want to assign or compare incompatible types, that is only possible by writing the required conversion functions (or doing nasty pointer casting etc., which pokes in the eye)

I wish more people saw it like that.

Quote
This is a feature of a strongly typed language. And yet, I would prefer C to be even more strongly typed. Implicit conversions are classic footguns. I don't want any more of them, and oh boy how much I struggled with PHP and its dynamic typing system (although PHP is a disaster in every other regard, too).

I like to point out that in C the variables are strongly typed, whereas the values (i.e. what's out there in the memory) is untyped. That's how you can cast a car to be a camel.

In other languages the variables are untyped whereas the values are strongly typed. Smalltalk is one example. There's no way you can cast a car to be a camel, but you can't predict which it ought to be.

Both approaches have their strengths and weaknesses.

Some languages have both, which - if well conceived - can have the advantages of both. Java is one.

Quote
Trying to make typing a bit faster is not a valid target. Less than 1% of time in programming is spent actually writing characters on code editor screen.

Exactly, in spades. Only beginners and amateurs cannot comprehend that.

If variables are strongly typed then the development environment has a key advantage: you can type ctrl-space (or whatever) and the developer will be presented with a list of what is legal in the context. If there is only one alternative, then it is literally spelled out for the developer. That saves many many keystrokes, and destroys the concept that keystrokes are an issue.
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 02, 2022, 09:21:58 am
I like to point out that in C the variables are strongly typed, whereas the values (i.e. what's out there in the memory) is untyped. That's how you can cast a car to be a camel.

This weird mix of safety and unsafety is also what makes C powerful and popular: it offers decent level of safety (camel = car is not possible), but also easy enough mechanisms to bypass that safety: camel = *((*camel_t)&car) (or the same via union type punning since C99 or so*). But the latter still - usually! - happens on purpose. Much better than by accident.

*) which is much better, since this "treat memory as the other type" conversion is now documented in the type itself:
typedef union
{
     struct
     {
         int n_humps; // usual camel things
     };
     car_t camel_as_car;
} camel_t;


There are and always will be three classes of people:
* Those who think C is too unsafe and too weakly typed, and should not allow bypassing type safety with any syntax,
* Those who think C is too tedious to work with with all that type bullshit ("come on, I don't want to type, why can't compiler do it for me?")
* Those who are relatively satisfied with the compromise.

It's worth understanding the first two are actually polar opposite of each other. Sometimes you can see a "C complainer" complain about these two opposite problems in the same post, which makes it look... schizophrenic. Or just that sometimes people just want to complain, without having any idea what they are talking about. (And I'm not referring to anyone particular this time.)
Title: Re: A microcontroller programming language
Post by: brucehoult on December 02, 2022, 11:12:42 am
Code: [Select]
// ah crud someone banned the following
// camel = *(camel_t*)&car;
extern camel *car_to_camel(car_t *car);
camel = *(car_to_camel(&car));

Code: [Select]
car_to_camel:
camel_to_car:
camel_to_horse:
horse_to_camel:
horse_to_car:
car_to_horse:
  ret
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 02, 2022, 02:37:19 pm
In c++ you can already do that, with slightly different syntax:

Code: [Select]
MyStructure a();
...
a = MyStructure();

And there is also nullptr:

Code: [Select]
some_pointer = nullptr;

Well that's certainly an improvement!
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 02, 2022, 02:59:13 pm
Initialization is indeed exception to the rule. Such is life, rules are arbitrary and not always perfectly logical. That does not mean rules are stupid or useless. I don't mind typing a bit of extra information, when it conveys important information for us programmers, like the type. This adds safety; the C standard requiring me to type the type (pun intended), means:

* I need to keep my brain enabled, less likely to do a mistake with the type
* If I make a mistake and use a wrong type, the compiler will check it for me and error out
* If I want to assign or compare incompatible types, that is only possible by writing the required conversion functions (or doing nasty pointer casting etc., which pokes in the eye)

This is a feature of a strongly typed language. And yet, I would prefer C to be even more strongly typed. Implicit conversions are classic footguns. I don't want any more of them, and oh boy how much I struggled with PHP and its dynamic typing system (although PHP is a disaster in every other regard, too).

Trying to make typing a bit faster is not a valid target. Less than 1% of time in programming is spent actually writing characters on code editor screen.

Well the inability of C to deduce (I use that word intentionally because there is no uncertainty) the type when we assign a value {0} is nothing to do with strong or weak typing. The type of the target is known and fixed, at compile time.

I'm not saying the rules are stupid or useless but at best its idiosyncratic and does make the language look almost like a toy language.

If being explicit about the type is so helpful then why do we not see this on every assignment:

Code: [Select]

int a;
short b;
long c;
char d;

a = (int)100;
b = (short)100;
c = (long)100;
d = (long)100;


But we do not see that in code, and I cannot see how such a practice - if it were imposed - would be anything other than unbearable frankly, what you're describing here is not strong typing, it is better described as training wheels!

It's absolutely appropriate to bring a runtime conversion risk to the developer's attention in cases where there is risk, but in the case of my_structure = {0} there is absolutely no risk (so far as I can see), what mistake could one make in this example?

Anyway as I said it is what it is, C is renowned for having many edge cases, it isn't pretty but it is what it is, if one is committed to using C then one accepts it for what it is and use the best discipline one can, but when considering ideas about new languages these observations are pertinent.


Title: Re: A microcontroller programming language
Post by: tggzzz on December 02, 2022, 03:14:42 pm
I like to point out that in C the variables are strongly typed, whereas the values (i.e. what's out there in the memory) is untyped. That's how you can cast a car to be a camel.

This weird mix of safety and unsafety is also what makes C powerful and popular: it offers decent level of safety (camel = car is not possible), but also easy enough mechanisms to bypass that safety: camel = *((*camel_t)&car) (or the same via union type punning since C99 or so*). But the latter still - usually! - happens on purpose. Much better than by accident.

I don't care when an error is caught, so long as it is caught. However, I prefer errors to be caught during compilation.

Quote
There are and always will be three classes of people:
* Those who think C is too unsafe and too weakly typed, and should not allow bypassing type safety with any syntax,
* Those who think C is too tedious to work with with all that type bullshit ("come on, I don't want to type, why can't compiler do it for me?")
* Those who are relatively satisfied with the compromise.

It's worth understanding the first two are actually polar opposite of each other. Sometimes you can see a "C complainer" complain about these two opposite problems in the same post, which makes it look... schizophrenic. Or just that sometimes people just want to complain, without having any idea what they are talking about. (And I'm not referring to anyone particular this time.)

I think there is an irregular verb adjective lurking there: I am safe, you are trustworthy, they are dangerous. Except I can be trusted not to trust myself.

In a low level language with only small applications and simple small libraries, it isn't too difficult to contain the dangers inherent in C.

As applications become larger and more complex, and there are many poorly-understood libraries, my preference to to catch as many type errors during compilation, and to allow casting but verify safety at runtime. That isn't C, of course. It doesn't mean C is wrong, just that there are better languages for large complex applications.

It is not acceptable to advocate minimising typing at the expense of not indicating/documenting intent. The time spent typing is trivial.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 02, 2022, 03:45:56 pm
Anyway, I've been immersed in the lexical area once more, I now realize that this is more important than I initially assumed, yes its important but I never really thought about revisiting the entire subject area and taking a fresh look at things.

Here are the kinds of literals that I think one could commit to in a new language:

Code: [Select]

// fixed point decimal literals

1               
11             
101010       
010101       
100 000       
100_000     
256:D         
123.45         

// fixed point binary literals

1:B           
101.101:B   
001 110 010:B
101 110.110:b
110_001:B
11 01101 01.1:B

// fixed point hexadecimal literals 

E:H
17FA:H
0AC0 7F4A:H
1AC0_7F4A:h
4AC6.7C3:H

// Octal literals (no justification/use case for octal fractions...?)

274:O
11 223 752:O
11_223_752:O

// floating point hex literals - 'p' (or 'P') eliminates the need for base designator

123.FE6p+3
123 334.FE6P-3
E05_22B.02p2

// floating point binary literals - 'b' (or 'B') eliminates the need for base designator

110.00101b+4
110.00101b3
1.0101101b-2


So here the (fixed point) base designator (which can be upper/lower case) is moved to the right end of the token, the last component, no need for the 0x prefix any more. No base designator is the same as having the decimal designator. Separators after a decimal point are not permitted (seem to serve no useful purpose IMHO). This is primarily based on a review of the (pretty broad) subject and looking at how some other languages notate these things.

Without the presence of an exponent indicator, fixed point is inferred.












Title: Re: A microcontroller programming language
Post by: Siwastaja on December 02, 2022, 05:07:11 pm
Well the inability of C to deduce (I use that word intentionally because there is no uncertainty) the type when we assign a value {0} is nothing to do with strong or weak typing. The type of the target is known and fixed, at compile time.

Sure is, but target is target! It's different thing! Did you not realize you can assign variables of different types to each other if they are compatible and have conversions? For example, uint32_t can be assigned into uint8_t, because narrowing conversion exists.

Quote
If being explicit about the type is so helpful then why do we not see this on every assignment:

The problem with yourself is, because you don't read, you never learn. I already mentioned that 0 is int, this is in language specification, but you ignored this and wasted our time to provide the stupid pseudo example I wanted to avoid by mentioning this early on.

This is why you don't need to specify the type of the literal 100, it's int by default. You can however modify the type, for example 100ULL is unsigned long long int. And you can assign int into float with built-in conversion, so you can write float a = 100. Isn't C fancy?

But what would {0} be? Would it be {int}, or maybe {int, int}, or how about {int, int, int}? Tell me. (And no, "same type as target of assignment" is not the answer.)

So C has already rules how to interpret literals, and how to convert between types. But it has no rule how to interpret more complex compound literals based on some other nearby code.
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 02, 2022, 05:40:14 pm
Quote
If being explicit about the type is so helpful then why do we not see this on every assignment:
The problem with yourself is...
this is going to be a looong thread :popcorn:
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 02, 2022, 05:49:32 pm
Quote
If being explicit about the type is so helpful then why do we not see this on every assignment:
The problem with yourself is...
this is going to be a looong thread :popcorn:

Likely not, I don't usually play these games for too long, it's getting near to the point of getting nothing out of it.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 02, 2022, 05:53:46 pm
Well the inability of C to deduce (I use that word intentionally because there is no uncertainty) the type when we assign a value {0} is nothing to do with strong or weak typing. The type of the target is known and fixed, at compile time.

Sure is, but target is target! It's different thing! Did you not realize you can assign variables of different types to each other if they are compatible and have conversions? For example, uint32_t can be assigned into uint8_t, because narrowing conversion exists.

Quote
If being explicit about the type is so helpful then why do we not see this on every assignment:

The problem with yourself is, because you don't read, you never learn.

Please, can you not disagree without being condescending? disparaging?

I already mentioned that 0 is int, this is in language specification, but you ignored this and wasted our time to provide the stupid pseudo example I wanted to avoid by mentioning this early on.

This is why you don't need to specify the type of the literal 100, it's int by default. You can however modify the type, for example 100ULL is unsigned long long int. And you can assign int into float with built-in conversion, so you can write float a = 100. Isn't C fancy?

I think it might actually be you who overlooked something, not I. One can write:

Code: [Select]

char number;

number = 100; // no need to prefix with (char)...an implicit conversion will be inserted.


So what is it? is the 100 a char or an int? Yes, it IS an int and it is being assigned to a char without a (char) on the basis of your earlier argument about "safety" and "footguns" I must assume that you always insert the (char) in such cases, if not why not?

But what would {0} be? Would it be {int}, or maybe {int, int}, or how about {int, int, int}? Tell me. (And no, "same type as target of assignment" is not the answer.)

It could have been whatever the lvalue's type is, or empty braces could have been reserved for just such a use.

So C has already rules how to interpret literals, and how to convert between types. But it has no rule how to interpret more complex compound literals based on some other nearby code.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 02, 2022, 06:00:28 pm
Much of what I say on this thread is simply my personal view on programming languages strengths and weaknesses.

Of course some will agree or disagree on this or that point, disagreement is absolutely a natural part of such conversation, I might be right, I might be wrong or it may be just a difference of opinion.

But there is nothing to be gained by elevating a disagreement a difference of opinion, to the level of personally disparaging remarks about one another.

Time, life is far too fleeting to be wasted in a state of angst, especially over such a relatively unimportant subject in the grand scheme of the universe.



Title: Re: A microcontroller programming language
Post by: tggzzz on December 02, 2022, 06:31:36 pm
If being explicit about the type is so helpful then why do we not see this on every assignment:

The problem with yourself is, because you don't read, you never learn.

Yes, everything does indicate that.

Now there's nothing wrong with not having read everything; we all start out as beginners. Nor is there anything wrong with going through some of the same thought processes as people have done in the past.

But not knowing the literature, not bothering to read things people suggest would advance his understanding and thinking, and still expecting other people to "discuss" and "explain" things is bad form.

Likely not, I don't usually play these games for too long, it's getting near to the point of getting nothing out of it.

I, and some others, decided that a while ago. The only reason I bother to look at this thread is that some of the peripheral things mentioned by other people are interesting.

But, hey, go knock yourself out :)
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 02, 2022, 07:14:23 pm
So what is it? is the 100 a char or an int? Yes, it IS an int and it is being assigned to a char without a (char) on the basis of your earlier argument about "safety" and "footguns" I must assume that you always insert the (char) in such cases, if not why not?

That's actually a good question. Yes, sometimes I write explicit cast, sometimes not. The reason not to is when it is obvious what is going to happen anyway (I assume the reader knows the basics of the C standard, but don't assume they know every detail by heart). Such case is assigning uint8_t to uint32_t: it is obvious that the input range [0..255] maps to output range [0..255] and extra bits get filled with zeroes.

On the other hand, maybe I want to emphasize the fact extra bits are generated. Not to compiler, but human reader. Then why not add the explicit cast.

Or, when I convert a float to integer, let's say:
double asdf = 123.4;
int val = asdf + 0.5; // round up and truncate after decimal point

One can emphasize the conversion by writing:
int val = (int)(asdf + 0.5);

But, you can just write
int val = asdf + 0.5;

because the implicit conversion does the same.

But explicit makes sense, because the same thing could be used in a more complex expression, where adding the cast does make the difference:
double something_else = (double)(int)(asdf+0.5) * qwerty;

Remember, programming is not only about making compiler happy, but to convey the meaning to human readers, too.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 02, 2022, 07:15:08 pm
If being explicit about the type is so helpful then why do we not see this on every assignment:

The problem with yourself is, because you don't read, you never learn.

Yes, everything does indicate that.

Now there's nothing wrong with not having read everything; we all start out as beginners. Nor is there anything wrong with going through some of the same thought processes as people have done in the past.

But not knowing the literature, not bothering to read things people suggest would advance his understanding and thinking, and still expecting other people to "discuss" and "explain" things is bad form.

Likely not, I don't usually play these games for too long, it's getting near to the point of getting nothing out of it.

I, and some others, decided that a while ago. The only reason I bother to look at this thread is that some of the peripheral things mentioned by other people are interesting.

But, hey, go knock yourself out :)

What exactly have I written that you regard as untrue? If you disagree with something I said then simply quote what I wrote and present your case without making disparaging insinuations. Trying to change the subject from programming language design to me does not help make any kind of logical case about language theory. Attacking the person, their presumed shortcomings does nothing to strengthen any position you might have, arguing emotionally won't  get you anywhere.

Seriously, I expected a higher standard intellectual of rigor in an engineering oriented forum.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 02, 2022, 07:25:50 pm
So what is it? is the 100 a char or an int? Yes, it IS an int and it is being assigned to a char without a (char) on the basis of your earlier argument about "safety" and "footguns" I must assume that you always insert the (char) in such cases, if not why not?

That's actually a good question. Yes, sometimes I write explicit cast, sometimes not. The reason not to is when it is obvious what is going to happen anyway (I assume the reader knows the basics of the C standard, but don't assume they know every detail by heart). Such case is assigning uint8_t to uint32_t: it is obvious that the input range [0..255] maps to output range [0..255] and extra bits get filled with zeroes.

On the other hand, maybe I want to emphasize the fact extra bits are generated. Not to compiler, but human reader. Then why not add the explicit cast.

Or, when I convert a float to integer, let's say:
double asdf = 123.4;
int val = asdf + 0.5; // round up and truncate after decimal point

One can emphasize the conversion by writing:
int val = (int)(asdf + 0.5);

But, you can just write
int val = asdf + 0.5;

because the implicit conversion does the same.

But explicit makes sense, because the same thing could be used in a more complex expression, where adding the cast does make the difference:
double something_else = (double)(int)(asdf+0.5) * qwerty;

Remember, programming is not only about making compiler happy, but to convey the meaning to human readers, too.

I don't disagree, if something can improve readability I'm all for it, but that's not what I've been arguing. What I said - and it is true - is that deducing the type when assigning {0} to some target is no more unreasonable or risky than deducing the type when I assign a short to a char, in fact it carries less risk.

I really cannot see nor has anyone convincingly explained, the theoretical justification to treat these cases differently, making one of these optional and the other obligatory - especially when the former carries more risk:

Code: [Select]

char X;
Data Y;

X = (char)0;

Y = (Data){0};


If I am allowed to omit the (char) then logically I should also be allowed to omit the (Data), this is all I've been saying, by all means disagree but my argument is not from ignorance or lack of reading literature, it based on the asymmetry, edge case syndrome that the language suffers from.

Anyway the outcome of this for me, is that a universal default() language function/keyword would be a huge help in any language, helping to avoid such gnarly code.

Here is another person's view (https://qr.ae/pr7QUQ), someone with plenty of familiarity with the "literature".

I quote:

Quote
People get surprisingly defensive about it—perhaps it's Stockholm Syndrome?—but C simply isn't a well-designed language. The fact that the other popular alternatives are more complex and worse (C++? really?) doesn't change that. The fact that you know its edge cases well enough to get by (but probably not well enough to avoid a myriad sources of bugs and security vulnerabilities unique to the language) doesn't change it either.

Here's his blog/website (https://jelv.is/) too.






Title: Re: A microcontroller programming language
Post by: tggzzz on December 02, 2022, 07:37:20 pm
If being explicit about the type is so helpful then why do we not see this on every assignment:

The problem with yourself is, because you don't read, you never learn.

Yes, everything does indicate that.

Now there's nothing wrong with not having read everything; we all start out as beginners. Nor is there anything wrong with going through some of the same thought processes as people have done in the past.

But not knowing the literature, not bothering to read things people suggest would advance his understanding and thinking, and still expecting other people to "discuss" and "explain" things is bad form.

Likely not, I don't usually play these games for too long, it's getting near to the point of getting nothing out of it.

I, and some others, decided that a while ago. The only reason I bother to look at this thread is that some of the peripheral things mentioned by other people are interesting.

But, hey, go knock yourself out :)

What exactly have I written that you regard as untrue?
...

Thereby indicating you didn't read what I wrote.

QED.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 02, 2022, 08:04:20 pm
If being explicit about the type is so helpful then why do we not see this on every assignment:

The problem with yourself is, because you don't read, you never learn.

Yes, everything does indicate that.

Now there's nothing wrong with not having read everything; we all start out as beginners. Nor is there anything wrong with going through some of the same thought processes as people have done in the past.

But not knowing the literature, not bothering to read things people suggest would advance his understanding and thinking, and still expecting other people to "discuss" and "explain" things is bad form.

Likely not, I don't usually play these games for too long, it's getting near to the point of getting nothing out of it.

I, and some others, decided that a while ago. The only reason I bother to look at this thread is that some of the peripheral things mentioned by other people are interesting.

But, hey, go knock yourself out :)

What exactly have I written that you regard as untrue?
...

Thereby indicating you didn't read what I wrote.

QED.

Nor will I be reading anything else you write.

Title: Re: A microcontroller programming language
Post by: gf on December 02, 2022, 08:17:03 pm
Code: [Select]
char X;
Data Y;
X = (char)0;
Y = (Data){0};

If I am allowed to omit the (char) then logically I should also be allowed to omit the (Data),

The expression Y={0} were already syntactically wrong, because {0} is not a valid expression according to the language definition.

A philosophy of C is that each expression has its own intrinsic type which does not depend on the context in which the expression is used. Therefore {0} cannot be a valid expression, because no unique type can be deduced from {0} alone. OTOH, the integer literal expression 0 has an intrinsic type (which is int).

EDIT:
Note that (Data){0} is NOT a cast expression, but a compund literal. The (Data) is part of the literal.
OTOH, (char)0 is a cast expression (unary cast operator) which converts the integer literal 0 to type char.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 02, 2022, 09:06:02 pm
Code: [Select]
char X;
Data Y;
X = (char)0;
Y = (Data){0};

If I am allowed to omit the (char) then logically I should also be allowed to omit the (Data),

The expression Y={0} were already syntactically wrong, because {0} is not a valid expression according to the language definition.

A philosophy of C is that each expression has its own intrinsic type which does not depend on the context in which the expression is used. Therefore {0} cannot be a valid expression, because no unique type can be deduced from {0} alone. OTOH, the integer literal expression 0 has an intrinsic type (which is int).

EDIT:
Note that (Data){0} is NOT a cast expression, but a compund literal. The (Data) is part of the literal.
OTOH, (char)0 is a cast expression (unary cast operator) which converts the integer literal 0 to type char.

Well this also contains an invalid expression then too surely?

Code: [Select]

MyStructure datum = {0};


Logically, in any language I've ever used (including C but like over a decade ago) these were regarded as having identical semantics:

Code: [Select]

short X = 0;

short Y; Y = 0;


Now if they don't (which seems to be the case for {0}) then that's fine, all well and good, but isn't a good language characteristic, this has been my argument all along about this, it's just - not good, these edge cases are the kind of things programming languages are compared on, symmetry, orthogonality, easy to reason about and so on. Unless one knew already that this asymmetry is there one would reason incorrectly.






Title: Re: A microcontroller programming language
Post by: tggzzz on December 02, 2022, 09:20:10 pm
If being explicit about the type is so helpful then why do we not see this on every assignment:

The problem with yourself is, because you don't read, you never learn.

Yes, everything does indicate that.

Now there's nothing wrong with not having read everything; we all start out as beginners. Nor is there anything wrong with going through some of the same thought processes as people have done in the past.

But not knowing the literature, not bothering to read things people suggest would advance his understanding and thinking, and still expecting other people to "discuss" and "explain" things is bad form.

Likely not, I don't usually play these games for too long, it's getting near to the point of getting nothing out of it.

I, and some others, decided that a while ago. The only reason I bother to look at this thread is that some of the peripheral things mentioned by other people are interesting.

But, hey, go knock yourself out :)

What exactly have I written that you regard as untrue?
...

Thereby indicating you didn't read what I wrote.

QED.

Nor will I be reading anything else you write.

That's consistent.

It would have been more precise if you had written "... be starting to read..."
Title: Re: A microcontroller programming language
Post by: gf on December 02, 2022, 09:26:51 pm
Well this also contains an invalid expression then too surely?

Code: [Select]
MyStructure datum = {0};

In conjunction with the declaration of datum, the ={0} is an initialization, and {0} is still not an expression, but an initializer-list enclosed in braces.
See https://en.cppreference.com/w/c/language/initialization
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 02, 2022, 09:43:14 pm
The lexical analysis for the space separated literals is looking to be done, it has a subtlety that means it cannot be handled by a pure FSM, a stack is required and that means also the this cannot be represented by a regular expression, now I don't know off hand if ANTLR supports that, it might because its described as not being a pure DFA based system.

Anyway I have this working, I have a previously written .Net text-table driven lexical analyzer and it already supports a stack so this was not too much effort.

Anyway, although the grammar is not cast in stone, it has taken shape and now this kind of thing is what it will start to look like:

Code: [Select]

proc initialize (count)
{
   arg count bin(8,2); // fixed point binary, 6.2 format
   dcl register_mask bin(24,8);
   dcl temporary dec(15,3); // fixed point decimal, 12.3 format

   count = 110011.01B;

   temporary = 23 000 000 000.25;

   register_mask = 2F03 D7F8.2:H;

   // Or equivalently

   register mask = 10 1111 0000 0011 1101 0111 1111 1000.00100000:b;

}


Of course underscore's are permitted, not just spaces, but the space/underscore must be a single space/underscore to avoid unsightly stuff developing.









   

Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 02, 2022, 09:44:26 pm
Quote
If being explicit about the type is so helpful then why do we not see this on every assignment:
The problem with yourself is...
this is going to be a looong thread :popcorn:
Likely not, I don't usually play these games for too long, it's getting near to the point of getting nothing out of it.
i mean... there are few members in this forum that are persistent enough to update their tiny progress/thought in their own "log thread" for extended amount of time, grow into hundreds of pages... and then later post the latest update saying the previous posts was a mistake... in the end i only look at the thread when somebody else made a new reply in case they provide new informations... to put insult to the pain... esp "language" is a really subjective matter imho. cheers.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 02, 2022, 09:50:41 pm
Well this also contains an invalid expression then too surely?

Code: [Select]
MyStructure datum = {0};

In conjunction with the declaration of datum, the ={0} is an initialization, and {0} is still not an expression, but an initializer-list enclosed in braces.
See https://en.cppreference.com/w/c/language/initialization

Yes, what you say is likely absolutely the case, that doesn't justify the design though, in the case of initializing a declaration, populating the lvalue is what happens and there's no need I can see for distinguishing between an lvalue used when we declare vs an lvalue used as the source expression of an assignment.

What I can "do to" the memory associated with some variable shouldn't differ between declaring it and assigning to it, IMHO anyway.



Title: Re: A microcontroller programming language
Post by: gf on December 02, 2022, 10:42:59 pm
Well this also contains an invalid expression then too surely?

Code: [Select]
MyStructure datum = {0};

In conjunction with the declaration of datum, the ={0} is an initialization, and {0} is still not an expression, but an initializer-list enclosed in braces.
See https://en.cppreference.com/w/c/language/initialization

Yes, what you say is likely absolutely the case, that doesn't justify the design though, in the case of initializing a declaration, populating the lvalue is what happens and there's no need I can see for distinguishing between an lvalue used when we declare vs an lvalue used as the source expression of an assignment.

What I can "do to" the memory associated with some variable shouldn't differ between declaring it and assigning to it, IMHO anyway.

The newer stuff is even orthogonal insofar as MyStructure datum = (MyStructure){0}; works as well, and here (MyStructure){0} is indeed an expression which can be also used as RHS of an assignment operator. The "traditional" declaration+initialization syntax MyStructure datum = { initializer-list }; dates back to K&R C, and was obviously retained for compatibility. Typed compound literals were introduced much later.

Still the right hand side of the binary assignment operator must be an expression, and {0} cannot be an expression. If it were a valid expression, then for instance

Code: [Select]
typedef struct { int x; } Data;

int f()
{
    return {0}.x;
}

would be valid as well. But of course this can't work when {0} has no associated type.

OTOH, with a properly typed compound literal (which is a valid expression) it works as expected, and f() returns 0:

Code: [Select]
typedef struct { int x; } Data;

int f()
{
    return (Data){0}.x;
}
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 03, 2022, 07:45:32 am
OTOH, with a properly typed compound literal (which is a valid expression) it works as expected, and f() returns 0:

Code: [Select]
typedef struct { int x; } Data;

int f()
{
    return (Data){0}.x;
}

Exactly. You can come up with more examples like this - the point is, the grammar and syntax is logical and repeatable.

There already is one "exception" to this logic, namely initializer list being a different animal from compound literal, and not needing a type.

One could argue C should add more of same type of exceptions, like the "in case of assignment to a struct type, compound literal is assumed to have the type of assignment target, unless otherwise explicitly typed". OK, then what? Now we have introduced more rules to the language, and what did we achieve? We saved a few keystrokes, hid the type from view, and you STILL need to write the type in all other forms of expressions.

Such thinking is very superficial. Again, the number of keystrokes needed is unimportant. Having to type something explicitly is a "problem" only for total beginners, who think majority of time spent in programming is hitting the keys. And any of this has absolutely no fundamental effect on the language design. More fundamental is the fact that these expressions have a type which is known at compile time, to both the programmer and the compiler.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 03, 2022, 01:57:39 pm
OTOH, with a properly typed compound literal (which is a valid expression) it works as expected, and f() returns 0:

Code: [Select]
typedef struct { int x; } Data;

int f()
{
    return (Data){0}.x;
}

Exactly. You can come up with more examples like this - the point is, the grammar and syntax is logical and repeatable.

There already is one "exception" to this logic, namely initializer list being a different animal from compound literal, and not needing a type.

One could argue C should add more of same type of exceptions, like the "in case of assignment to a struct type, compound literal is assumed to have the type of assignment target, unless otherwise explicitly typed". OK, then what? Now we have introduced more rules to the language, and what did we achieve? We saved a few keystrokes, hid the type from view, and you STILL need to write the type in all other forms of expressions.

Such thinking is very superficial. Again, the number of keystrokes needed is unimportant. Having to type something explicitly is a "problem" only for total beginners, who think majority of time spent in programming is hitting the keys. And any of this has absolutely no fundamental effect on the language design. More fundamental is the fact that these expressions have a type which is known at compile time, to both the programmer and the compiler.

This discussion about {0} and the asymmetric way it's handled, combined with the serious problem of it not satisfying the grammar for an expression, is IMHO a good example of the language's many shortcomings, shortcomings that are very hard to address and to improve.

This is also an example of why I place great importance on a grammar, once a grammar is cast in stone it is very hard indeed to change it much, any newer, changed grammar must be able to still consume the earlier grammar.

I've been reading about the Julia language, that is actually refreshingly interesting. It dispenses with the braces as block delimiters, as does Python, but does not rely on indentation like Python (a disaster waiting to happen IMHO). It also dispenses with semicolons, these two features do give it a visual clarity that's nice too look at, but I think it loses an ability to wrap statements across multiple lines, but that's really not a huge deal in this day and age (the semicolon likely originated at a time when lines were 80 chars max).
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 03, 2022, 04:15:14 pm
If semicolons are discarded as statement terminators and line endings adopted instead, and we lose the braces we get something more compact, here's what the earlier fragment becomes:

Code: [Select]
proc initialize (count)

   arg count bin(8,2)  // fixed point binary, 6.2 format
   dcl register_mask bin(24,8)
   dcl temporary dec(15,3)  // fixed point decimal, 12.3 format

   count = 110011.01B

   temporary = 23 000 000 000.25

   register_mask = 2F03 D7F8.2:H

   // Or equivalently

   register mask = 10 1111 0000 0011 1101 0111 1111 1000.00100000:b

end


Semicolons basically become optional, because it and CR/LF are treated in the same way during parsing. Because we have keywords for all executable statements, they can serve the same roles as { and to replace the role of } we just have a new "end" keyword:

Code: [Select]
proc initialize (count)

   if (a = 23)
      a = 0
   end

   if (a = 55)
      a = 10
   end

   if (a > 1024)
      a = 0
      call reset(100)
   end

   if (a < 2048)
      a = 50
      call restart(0)
   else
      a = 1024
   end

   while (state = READY)
        call refresh_display(250)
        state = get_state()
   end

end


PL/I used do; and end; for { and }, the braces are an improvement, much less text but being able to eliminate the braces altogether seems very appealing, it works fine in Julia.

Here's someone's blog post I stumbled upon when reading about Julia, all about this subject The Case Against Curly Braces {} Languages (https://erik-engheim.medium.com/the-case-against-curly-braces-languages-dd0a55840fcb).


Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 03, 2022, 08:40:44 pm
Such thinking is very superficial. Again, the number of keystrokes needed is unimportant. Having to type something explicitly is a "problem" only for total beginners, who think majority of time spent in programming is hitting the keys.

Oh yeah, I keep saying this too.
Focusing on code compactness if futile and unfortunately an obsession not only for beginners. It seems very widespread. Of course, it's much easier to tackle than the more fundamental and more important concepts, which is inherently hard.

The average "productivity" of a software developer is commonly considered about 100 lines of code per day. That means code that is meeting the requirements, fully debugged and tested. Typing 100 lines of code takes only a few minutes for someone with normal typing skills. Clearly this is an absolutely minor fraction of the time spent on developing software.

Time spent on fixing bugs coming from ambiguous code statements / ill-understood implicit language behaviors is, in comparison, gigantic.
Title: Re: A microcontroller programming language
Post by: pcprogrammer on December 04, 2022, 06:18:31 am
Code: [Select]
  if(a == b)
    c = 10
  end

Is basically just as much typing as

Code: [Select]
  if(a == b)
  {
    c = 10;
  }

I myself like the braces. Wish they had used them in verilog in the same way instead of the begin and end... statements. But that comes down to what one is used to.

And frankly this is all cosmetics, nothing to do with improving functionality of a programming language. So I agree with SiliconWizard that it is futile.
Title: Re: A microcontroller programming language
Post by: brucehoult on December 04, 2022, 06:51:48 am
Code: [Select]
  if(a == b)
    c = 10
  end

Is basically just as much typing as

Code: [Select]
  if(a == b)
  {
    c = 10;
  }

I myself like the braces. Wish they had used them in verilog in the same way instead of the begin and end... statements. But that comes down to what one is used to.

I like "end" better, if only because it makes for tidier syntax for signalling what is is you are ending, in a way that the compiler can check (i.e. not just a comment).

Code: [Select]
  line:
  while (myLine = readLine())
    :
    if (...) next line;
    :
    if (...) last line;
    :
  end while line

On the "end" statement, either the "while" or the label can be omitted: "end", "end while", "end line".

Obviously for C/Java it is continue/break not next/last but I quite like that Perl idea.

Title: Re: A microcontroller programming language
Post by: pcprogrammer on December 04, 2022, 07:21:06 am
I like "end" better, if only because it makes for tidier syntax for signalling what is is you are ending, in a way that the compiler can check (i.e. not just a comment).

........

On the "end" statement, either the "while" or the label can be omitted: "end", "end while", "end line".

I see your point, but then it should be done like in verilog where it uses the type within the end, like endcase and endmodule. Only it is not completely consistent because for instance an if uses plain begin / end to mark the block of code.

And to ensure maintained readability it should not be optional. So single line if statements should still end with an endif. On the other hand, the loose rules of C are very nice, but can result in code that is very hard to read.

It all boils down to personal preference :)
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 04, 2022, 09:30:27 am
what is better than Basic? in term of cosmetic and human most friendly and sensible syntax?
Code: [Select]
If (a = b) Then
    c = 10
End If
the problem with it is it doenst support pointer and true OOP.. if Basic can do whatever C can, i think i've left C behind too long time ago...
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 04, 2022, 09:52:52 am
Code: [Select]
If (a = b) Then
    c = 10
End If

That hurts my (somewhat pedantic and logical) eye because the same operator (=) is used for two different purposes. I don't mind using = as comparison operator, but then something else is needed for assignment, such as :=, or, borrowing from mathematics, "let c = 10".
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 04, 2022, 02:48:32 pm
Code: [Select]
  if(a == b)
    c = 10
  end

Is basically just as much typing as

Code: [Select]
  if(a == b)
  {
    c = 10;
  }

I myself like the braces. Wish they had used them in verilog in the same way instead of the begin and end... statements. But that comes down to what one is used to.

And frankly this is all cosmetics, nothing to do with improving functionality of a programming language. So I agree with SiliconWizard that it is futile.

I understand you, I used PL/I for years and that used do; and end; (with optional labels like Bruce was describing) and I when I began to use C in earnest, appreciated the simplicity and elegance of that. So until literally a couple days ago I was pleased to try and define a grammar that's a hybrid of PL/I with some stuff from other languages including the braces from C.

I've looked at quite a few other language grammars recently, and discovered that Python (like F#) regards indentation as having syntactic meaning, that's kind of interesting but frankly the ease with which valid code can chance behavior/meaning just by a copy/paste error (spaces/tabs unexpectedly creeping in and out), is too large IMHO.

Now that could be made to work in principle, if one could encode the number of spaces inside the line's text using non-displayable Unicode or something, but it wouldn't fly really without special editing tools.

Then I came across Julia and noticed that it is a pretty respected language, truly interesting and novel. Like Python its a "dynamic" language but it is a different beast altogether.

I'd never thought about this before but one can use a keyword statement's keyword to designate the start of a "block" and then require simply an "end" to close it. In PL/I we do use the keyword but only if its part of what they call a "do group" (basically do-group means something than can represent a block, like "do while" and "do until" and "if X then do;" etc.). Pascal is similar with it's reliance on "begin".

But Julia just uses the keyword alone (OK this requires and end always) which in the cold light of day is pretty intelligent of them - IMHO. Then of course Julia also dispenses with semicolons as statement terminators, that puzzled me at first but I can see now that it is quite legitimate to do that.

You say "it is all cosmetic" but decisions to need to be made, one cannot have an informally defined grammar, so one must make decisions if one is considering designing a new language. You can't say "don't change that" because what is that relative too? I'm discussing a new language not changing an existing one! Even if one copies or largely borrows another grammar one must still choose which!

You also say "improving functionality of a programming language" but I am not doing that ! I'm looking at how a brand new language will look, and decisions about block structuring, keywords, statement syntax - (i.e. grammar!) must be made.







Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 04, 2022, 03:00:22 pm
Code: [Select]
If (a = b) Then
    c = 10
End If

That hurts my (somewhat pedantic and logical) eye because the same operator (=) is used for two different purposes. I don't mind using = as comparison operator, but then something else is needed for assignment, such as :=, or, borrowing from mathematics, "let c = 10".

I understand you, but allowing and assignment to also be an expression is illegal, meaningless in many programming languages. Derivatives of C do it because, well, because they're derivatives of C! It is in fact one of my biggest complaints about these languages actually.

So unless one has only ever used C-like languages, the code:

Code: [Select]

If (a = b)
    c = 10
end


Will look completely run of the mill, no confusion whatsoever. If one does - as you said - object to the same symbol being used then yes it is the assignment symbol that should be different not the equality symbol, no mathematician would ever tolerate :

a = a + 10

for example, because it is untrue, a provably false proposition. So a purist could argue for a notation more like:

Code: [Select]

If (a = b)
    c <= 10
end


And then "=" is always part of an expression just as its ">" or "<" leading to improved symmetry.

I've been thinking about this in fact but have leaned towards the fact that "=" for assignment is so ubiquitous, like, almost all languages have adopted that, but it should be considered, perhaps another symbol is the right thing to do...
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 04, 2022, 03:56:13 pm
Code: [Select]
If (a = b) Then
    c = 10
End If
That hurts my (somewhat pedantic and logical) eye because the same operator (=) is used for two different purposes. I don't mind using = as comparison operator, but then something else is needed for assignment, such as :=, or, borrowing from mathematics, "let c = 10".
we are aware of this, but the compiler is clever enough to know when to assign and when to compare. it also happen to us human, there is difference when we say "a equals b" and "if a equals b", thats why i said, language is too subjective. if you want to impose technically and grammatically correct syntax in communication, good luck!.

I understand you, but allowing and assignment to also be an expression is illegal, meaningless in many programming languages.
try to make your language same popularity with Basic at best... even though its grammatically incorrect, it can make good programs. proven millions time.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 04, 2022, 06:56:38 pm
Something else to consider in a language aimed at improving how we write code for MCUs, is the inclusion of operators for:

1. Arithmetic shifting
2. Logical shifting
3. Rotation
4. NAND
5. NOR

Some of these are available in some languages as core operators, but I don't think any language covers all of the possibilities (with the exception of assembler).

Title: Re: A microcontroller programming language
Post by: rstofer on December 04, 2022, 07:25:03 pm
Pascal uses := for assignment and = for comparison.  It took about 5 minutes to get used to this notation.  I rather like it...
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 04, 2022, 08:42:44 pm
Pascal uses := for assignment and = for comparison.  It took about 5 minutes to get used to this notation.  I rather like it...

Yeah this is fine and a no-brainer. I don't really see a point of trying to invent something else. It just works.
I think that was discussed in older threads too. I don't really like the "let" keyword and it's not for parsing or typing reasons. I personally think "let" is ill-used for normal assignment, as to me it should not be used to change values of objects but only to define them as an invariant - that's how it's usually used in maths.

So if I see something like:
Code: [Select]
let x = 1;
(...)
let x = x + 1;

It doesn't look right to me.
IMHO, if we want to find a use for a "let" keyword, it should preferably be used as a invariant definition. For instance (types are omitted):
Code: [Select]
x := 1;
let y = 2*x + 1;  // Could be ':=' here, open to discussion. Now 'y' holds '3'.
(...)
x := x + 1;  // When this assignment is completed, 'y = 2*x+1' still holds, so 'y' now holds '5'.

This feature looks pretty cool to me. The "let" keyword may not be the best either and could be replaced by something else more explicit, such as just "invariant".
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 04, 2022, 09:39:27 pm
how much time you "assign" vs time you "compare"? i think "assign" is much more used/typed... so assign using "=" and compare using "==" is much more sensible, that is C, and i dont have to keep pressing shift key so many times. https://www.jonathanshultsmd.com/blog/can-typing-cause-carpal-tunnel-syndrome (https://www.jonathanshultsmd.com/blog/can-typing-cause-carpal-tunnel-syndrome)
Title: Re: A microcontroller programming language
Post by: rstofer on December 05, 2022, 12:35:45 am
I always thought that 'let' came from BASIC and was used to allow the interpreter to understand the type of the next statement.  It soon became optional if, in fact, it was ever actually required.

Not a fan...
Title: Re: A microcontroller programming language
Post by: cfbsoftware on December 05, 2022, 01:21:42 am
how much time you "assign" vs time you "compare"? i think "assign" is much more used/typed... so assign using "=" and compare using "==" is much more sensible
Indeed. My Oberon code base shows ':=' (assign) is used 5 times more than '=' (compare). However, another factor to consider is the number of times a program is read rather than written. In my case this is often many thousands of times more.

Beginner Pascal / Oberon programmers are taught to pronounce := as "becomes equals".  Then

Code: [Select]
IF a = b

is read as "If a equals b ..."

and

Code: [Select]
a := b
is read as "a becomes equal to b"

I still haven't got the hang of how to read "=" and "==" in my head without it being awkward.
Title: Re: A microcontroller programming language
Post by: brucehoult on December 05, 2022, 05:24:05 am
how much time you "assign" vs time you "compare"? i think "assign" is much more used/typed... so assign using "=" and compare using "==" is much more sensible, that is C, and i dont have to keep pressing shift key so many times. https://www.jonathanshultsmd.com/blog/can-typing-cause-carpal-tunnel-syndrome (https://www.jonathanshultsmd.com/blog/can-typing-cause-carpal-tunnel-syndrome)

On my keyboard I need to press shift for all of { } ( )  * + ~ ! % ^ & " < > ? all of which are used heavily in programs. I don't think a few more : is going to make any difference.

And probably less than 5% of a programmer's time is spent typing code. We are thinkers, not data entry operators.
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 05, 2022, 05:52:58 am
However, another factor to consider is the number of times a program is read rather than written. In my case this is often many thousands of times more.
Irrelevant! As what we typed will not get translated/manifested into machine's language/code.. 'a = b' in c, and 'a := b' in pascal will end up as the same code in machine, interpreted/translated by c and pascal compiler respectively, compiler job is to bridge human friendly syntax into machine codes, the only difference is we have to type one more character in pascal..

I still haven't got the hang of how to read "=" and "==" in my head without it being awkward.
because your mother language (pascal) clouded your mind, make you harder to get around another language, esp a language as good a c :P... if we just follow a language's notion and grammar, rather than our heart desire.. there should be no problem.. but then just as you pick your fav language.. i also did and abandon pascal grammar for the above logical reason, and some other technical reasons.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 05, 2022, 12:47:01 pm
Pascal uses := for assignment and = for comparison.  It took about 5 minutes to get used to this notation.  I rather like it...

Yeah this is fine and a no-brainer. I don't really see a point of trying to invent something else. It just works.
I think that was discussed in older threads too. I don't really like the "let" keyword and it's not for parsing or typing reasons. I personally think "let" is ill-used for normal assignment, as to me it should not be used to change values of objects but only to define them as an invariant - that's how it's usually used in maths.

So if I see something like:
Code: [Select]
let x = 1;
(...)
let x = x + 1;

It doesn't look right to me.
IMHO, if we want to find a use for a "let" keyword, it should preferably be used as a invariant definition. For instance (types are omitted):
Code: [Select]
x := 1;
let y = 2*x + 1;  // Could be ':=' here, open to discussion. Now 'y' holds '3'.
(...)
x := x + 1;  // When this assignment is completed, 'y = 2*x+1' still holds, so 'y' now holds '5'.

This feature looks pretty cool to me. The "let" keyword may not be the best either and could be replaced by something else more explicit, such as just "invariant".

I think that's true, its used that way in F# and some other functional languages, denoting a value and not a variable (well real functional languages don't have variables anyway).
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 05, 2022, 01:13:55 pm
how much time you "assign" vs time you "compare"? i think "assign" is much more used/typed... so assign using "=" and compare using "==" is much more sensible
Indeed. My Oberon code base shows ':=' (assign) is used 5 times more than '=' (compare). However, another factor to consider is the number of times a program is read rather than written. In my case this is often many thousands of times more.

Beginner Pascal / Oberon programmers are taught to pronounce := as "becomes equals".  Then

Code: [Select]
IF a = b

is read as "If a equals b ..."

and

Code: [Select]
a := b
is read as "a becomes equal to b"

I still haven't got the hang of how to read "=" and "==" in my head without it being awkward.

Spare a thought for the Javascript developer who deals with = and == and === such is nature of modern grammars, frankly absurd in the cold light of day.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 05, 2022, 01:22:52 pm
A new language should also support UTF-8 source files too. Then the prospect of richer identifiers emerges and a huge range of interesting operators symbols too. Many modern languages are like this, C# and F# and many others allow Greek letters, the traditional division symbol and many others.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 05, 2022, 02:33:47 pm
Pascal uses := for assignment and = for comparison.  It took about 5 minutes to get used to this notation.  I rather like it...

Yup. Faster than to get used to than = and ==, and less prone to unnoticed errors.

The clearest assignment operator is, of course ← as in "x ← x + 1". Bonus points for recognising which language got that right many decades ago :)

As long as you don't have to give it a three-fingered salute, who cares whether one or two characters has to be typed. We're not using ASR33s anymore, are we mechatrommer?.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 05, 2022, 03:00:32 pm
Pascal uses := for assignment and = for comparison.  It took about 5 minutes to get used to this notation.  I rather like it...

Yup. Faster than to get used to than = and ==, and less prone to unnoticed errors.

The clearest assignment operator is, of course ← as in "x ← x + 1". Bonus points for recognising which language got that right many decades ago :)

As long as you don't have to give it a three-fingered salute, who cares whether one or two characters has to be typed. We're not using ASR33s anymore, are we mechatrommer?.

Ahh APL, of which I have fond memories from my early 1980s mainframe days!

Here's a great sight (https://tryapl.org/) for playing with APL, such an expressive language, here's the definition of a function to calculate the average of any number of integers:

Code: [Select]

avg ← { (+ / ⍵) ÷ ⍴ ⍵ }
 





Title: Re: A microcontroller programming language
Post by: tggzzz on December 05, 2022, 03:57:38 pm
Pascal uses := for assignment and = for comparison.  It took about 5 minutes to get used to this notation.  I rather like it...

Yup. Faster than to get used to than = and ==, and less prone to unnoticed errors.

The clearest assignment operator is, of course ← as in "x ← x + 1". Bonus points for recognising which language got that right many decades ago :)

As long as you don't have to give it a three-fingered salute, who cares whether one or two characters has to be typed. We're not using ASR33s anymore, are we mechatrommer?.

Ahh APL, of which I have fond memories from my early 1980s mainframe days!

Here's a great sight (https://tryapl.org/) for playing with APL, such an expressive language, here's the definition of a function to calculate the average of any number of integers:

Code: [Select]

avg ← { (+ / ⍵) ÷ ⍴ ⍵ }
 

I wasn't thinking of APL, since I'm not a fan and it hasn't been very influential. By and large APL seems to be a mistake carried through to perfection. There is one thing that APL did get right, the difference between "-3" and "¯3". But that is "correcting" a standard quirk with maths notation that requires every generation to learn about that mistake.

No, I was thinking of a language that has been extremely influential and is better than quite a few of its successors. That language's syntax can be easily described with examples on a single sheet of A4 paper. Very few languages manage anything close.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 05, 2022, 04:13:23 pm
Pascal uses := for assignment and = for comparison.  It took about 5 minutes to get used to this notation.  I rather like it...

Yup. Faster than to get used to than = and ==, and less prone to unnoticed errors.

The clearest assignment operator is, of course ← as in "x ← x + 1". Bonus points for recognising which language got that right many decades ago :)

As long as you don't have to give it a three-fingered salute, who cares whether one or two characters has to be typed. We're not using ASR33s anymore, are we mechatrommer?.

Ahh APL, of which I have fond memories from my early 1980s mainframe days!

Here's a great sight (https://tryapl.org/) for playing with APL, such an expressive language, here's the definition of a function to calculate the average of any number of integers:

Code: [Select]

avg ← { (+ / ⍵) ÷ ⍴ ⍵ }
 

I wasn't thinking of APL, since I'm not a fan and it hasn't been very influential. By and large APL seems to be a mistake carried through to perfection. There is one thing that APL did get right, the difference between "-3" and "¯3". But that is "correcting" a standard quirk with maths notation that requires every generation to learn about that mistake.

Well given how unreceptive, even hostile, many can be, to new languages and language ideas, radical departures from convention, it's no surprise that APL isn't appreciated as much as it could be. Most people are very conservative even in this day and age, they don't like change generally, not if it means they have to change and programming languages are particularly prone to this.

The degree to which a language is "influential" isn't wholly attributable to the languages innate qualities, it is often more of a reflection of other factors. APL was not initially anything to even do with computers, it was an exercise in developing a notation for teaching certain aspects of mathematics, only later did it dawn on them that it could be used for a language.

Iverson received a Turning Award for his work on notation, he showed how thought itself (https://www.jsoftware.com/papers/tot.htm) is often directed/restrained, even sometimes blinkered by, notation.

Consider too:

Quote
APL has formed the basis of, or influenced, the following languages:[citation needed]

A and A+, an alternative APL, the latter with graphical extensions.
FP, a functional programming language.
Ivy, an interpreter for an APL-like language developed by Rob Pike, and which uses ASCII as input.[46]
J, which was also designed by Iverson, and which uses ASCII with digraphs instead of special symbols.[7]
K, a proprietary variant of APL developed by Arthur Whitney.[8]
MATLAB, a numerical computation tool.[6]
Nial, a high-level array programming language with a functional programming notation.
Polymorphic Programming Language, an interactive, extensible language with a similar base language.
S, a statistical programming language (usually now seen in the open-source version known as R).
Speakeasy, a numerical computing interactive environment.
Wolfram Language, the programming language of Mathematica.[47]


So, MATLAB and Mathematica...

No, I was thinking of a language that has been extremely influential and is better than quite a few of its successors. That language's syntax can be easily described with examples on a single sheet of A4 paper. Very few languages manage anything close.

No idea, you'll have to tell me!



Title: Re: A microcontroller programming language
Post by: tggzzz on December 05, 2022, 08:59:22 pm
Just to tease you, here is a bowdlerised assessment of language X...

X is also one of the most influential programming languages. Virtually all of the object-oriented languages that came after—Flavors,CLOS, Objective-C, Java, Python, Ruby, and many others—were influenced by X. X was also one of the most popular languages for agile software development methods, rapid application development (RAD) or prototyping, and software design patterns. The highly productive environment provided by X platforms made them ideal for rapid, iterative development....

Many of the language and implementation concepts in X have become standard elsewhere. Some are so counterintuitive that it is necessary to inform the unenlightened that their simple undergraduate experience is grossly misleading.

X has 6 keywords.

By comparison APL was a deadend. 7-7-7 being 7 confuses most people :)
Title: Re: A microcontroller programming language
Post by: DiTBho on December 05, 2022, 09:07:00 pm
15 pages of ... sugar ...   :-BROKE
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 05, 2022, 09:10:37 pm
Here's a great sight (https://tryapl.org/) for playing with APL, such an expressive language, here's the definition of a function to calculate the average of any number of integers:

Code: [Select]

avg ← { (+ / ⍵) ÷ ⍴ ⍵ }
 

You call that expressive, I call that a fantastic exercise in obfuscation.
Apart from the assignment symbol that almost anyone with a programming experience will get, the right hand part is undecipherable.

I'm under the impression again that you are conflating expressiveness with the least number of keystrokes possible.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 05, 2022, 10:18:55 pm
Here's a great sight (https://tryapl.org/) for playing with APL, such an expressive language, here's the definition of a function to calculate the average of any number of integers:

Code: [Select]

avg ← { (+ / ⍵) ÷ ⍴ ⍵ }
 

You call that expressive, I call that a fantastic exercise in obfuscation.

Erm, well, Not to someone who understands APL, but perhaps you're right, I mean look at this, how obfuscated did Maxwell need to get, I mean, come on James!

(https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRMkBVjI_WDdDhB6XDcBfkacwMmPmqibRED9EhPlKm37I-RmEtZyT-om6R24kAAX2AlPU8&usqp=CAU)

Apart from the assignment symbol that almost anyone with a programming experience will get, the right hand part is undecipherable.

I'm under the impression again that you are conflating expressiveness with the least number of keystrokes possible.

Just as Maxwell's equations are undecipherable until one learns their meaning, lack of familiarity with a language is really not a very good argument that the language is obfuscatory.

The APL is incredibly simple once you become familiar with the notation. The + / ⍵ means "take every element of the vector ⍵ and insert the "+" operator in-between them all, then evaluate that expression. Then it says "divide that result by" ⍴ ⍵ (the "shape" of the array ⍵ ) which is just the number of elements in the array.







Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 05, 2022, 10:30:42 pm
The lack of reserved words in the grammar I'm developing, lends itself well to multiple languages.

These two source fragments generate identical parse trees:

Code: [Select]

function test binary(31)

   declare a string

   while true

      a = a + 1
   
   end

end



Code: [Select]

fonction test binaire(31)

   déclarer a chaîne

   pendant que true

      a = a + 1

   fin

fin


Note another very powerful feature, keywords can contain multiple terms, like the PL/I language allowed "goto" or "go to" the parser was fine with that idea. Same here, English "while" in French is "pendant que" presents no problems.

The only difference is that the first is compiled with an English JSON keyword dictionary and the French with a different JSON dictionary, the Lexical Analyzer and Parser are totally unaware that the sources are in different languages!

The technology I have here now, would easily support a language where we can specify the keyword language for different regions in a source file, not just have that fixed for the whole source file or for an entire compilation. One could mix or copy/paste a example algorithm from a Spanish engineer into code being written by a German and share that with a developer who works in English!

The real power here is that we can freely add a new keyword too all - any number of - languages and never ever lose backward compatibility for any source written in any language, if we had thirty dictionaries defined and we wanted to add a keyword "await" then we'd that to the English dictionary and add whatever it translates too to each of the other twenty nine dictionaries and we're done, even if some or all had used the term "await" for a variable or function name!

Time for some Latin motor control software!

Try doing that in C or C++ !



Title: Re: A microcontroller programming language
Post by: tggzzz on December 05, 2022, 11:06:32 pm
Why stop there (at keywords)?

Why not have the ability to add new control structures?

Why not be able to add them while the program is executing?

Why stop there? Define a new domain specific language? (Answer: you could, but a domain specific library is probably better)

Start from the right foundation, and all that is trivial.
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 05, 2022, 11:16:50 pm
Pascal uses := for assignment and = for comparison.  It took about 5 minutes to get used to this notation.  I rather like it...

Yup. Faster than to get used to than = and ==, and less prone to unnoticed errors.

The clearest assignment operator is, of course ← as in "x ← x + 1". Bonus points for recognising which language got that right many decades ago :)

As long as you don't have to give it a three-fingered salute, who cares whether one or two characters has to be typed. We're not using ASR33s anymore, are we mechatrommer?.
the fact is.. pascal is dead (technically), and you dont have anything to back it up, whether mechatrommers exist or not... whats grew on everybody today is Python, which uses the same assign and compare syntax as C... https://www.pythontutorial.net/python-basics/python-comparison-operators/ (https://www.pythontutorial.net/python-basics/python-comparison-operators/)
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 05, 2022, 11:22:25 pm
Pascal uses := for assignment and = for comparison.  It took about 5 minutes to get used to this notation.  I rather like it...

Yup. Faster than to get used to than = and ==, and less prone to unnoticed errors.

The clearest assignment operator is, of course ← as in "x ← x + 1". Bonus points for recognising which language got that right many decades ago :)

As long as you don't have to give it a three-fingered salute, who cares whether one or two characters has to be typed. We're not using ASR33s anymore, are we mechatrommer?.
the fact is.. pascal is dead (technically), and you dont have anything to back it up, whether mechatrommer exists or not... whats grew on everybody today is Python, which uses the same assign and compare syntax as C... https://www.pythontutorial.net/python-basics/python-comparison-operators/ (https://www.pythontutorial.net/python-basics/python-comparison-operators/)

Python relies on indentation to express containment/nesting, it is a horrible accident waiting to happen.

You can copy a large Python source file (say from some web site) and - inadvertently or intentionally - do nothing more than add/remove spaces and the logic can change hugely yet you'd not know it, it would run and likely appear fine, but could execute differently from what you thought you copied...

F# and some other functional languages also rely on indentation, but due to the fundamental differences between functional and imperative languages, corrupting spaces in F# will almost always (if not always) either have zero impact or cause compile errors.

Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 05, 2022, 11:43:20 pm
Python relies on indentation to express containment/nesting, it is a horrible accident waiting to happen.
learn from the lesson or else....
(https://imgs.xkcd.com/comics/standards.png)
do you really want to reinvent ASR33's age ":="? ;D
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 05, 2022, 11:50:48 pm
Python relies on indentation to express containment/nesting, it is a horrible accident waiting to happen.
learn from the lesson or else....
(https://imgs.xkcd.com/comics/standards.png)
do you really want to reinvent ASR33's age ":="? ;D

What standard did you use for counting how many standards there are...
Title: Re: A microcontroller programming language
Post by: cfbsoftware on December 05, 2022, 11:59:12 pm
I still haven't got the hang of how to read "=" and "==" in my head without it being awkward.
because your mother language (pascal) clouded your mind, make you harder to get around another language, esp a language as good a c :P... if we just follow a language's notion and grammar, rather than our heart desire.. there should be no problem.. but then just as you pick your fav language.. i also did and abandon pascal grammar for the above logical reason, and some other technical reasons.
I'm not sure what you mean by 'mother language' but Pascal certainly was not my 'first' language. I was taught FORTRAN IV at school in the 60's and ALGOL 60 at Uni in the early 70's and my programmer career started with Varian's TASC language.

I didn't start using Pascal until 1976. My C programming career started later in 1985 and was followed by 10 years of comparative misery and pain after which I promised myself I would never, ever, program in C again. I have kept that promise.

Although I liked Pascal it had its fair share of logical quirks, which I found a constant source of irritation. Fortunately these were elegantly eliminated in the early 80's when Wirth designed its successor: Modula-2. Any logical reasons for not using a language evaporated for me. Modula-2 became my language of choice when I was programming for 'fun'and not for a job.

Nowadays I am in the fortunate position of being able to use Modula-2's successor, Oberon, when programming for both business and pleasure. Being able to use it to program microcontrollers also allows me also to indulge in my lifelong passion of electronics. These are certainly the most satisfying days of my computing life. ;)
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 06, 2022, 05:32:26 am
The fun part is that the thread is starting to look like a popularity contest (which is what ends up happening in many programming language threads anyway) rather than stick to purely technical points. Which usually means game over, but I don't wanna ruin the thread. Good luck!
Title: Re: A microcontroller programming language
Post by: brucehoult on December 06, 2022, 08:26:46 am
The fun part is that the thread is starting to look like a popularity contest (which is what ends up happening in many programming language threads anyway) rather than stick to purely technical points. Which usually means game over, but I don't wanna ruin the thread. Good luck!
That happened long ago. The thread OP is still, as of yesterday, hurling personal abuse and insults at those who disagree with HIS pet ideas.
Title: Re: A new, hardware "oriented" programming language
Post by: Brianf on December 06, 2022, 10:49:59 am
What was PL/M like, and should it come back?  I've never used it.

As it looks like I'm the only one here who has used PL/M (for about 3 years) I'd put is somewhere between C and assembler. I still have a bit of a soft spot for it and would happily use it if a compiler were available. 

I read above that PL/M is a stripped back PL/I. Having never looked at any PL/I source code until today I can see a vague passing resemblance but that might be the all-caps text fooling me.
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 06, 2022, 12:18:29 pm
if i'm going to make a new language, it will be somewhere between Basic and C... and then we have 16 competing standards...
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on December 06, 2022, 02:36:56 pm
What was PL/M like, and should it come back?  I've never used it.

As it looks like I'm the only one here who has used PL/M (for about 3 years) I'd put is somewhere between C and assembler. I still have a bit of a soft spot for it and would happily use it if a compiler were available.

I read above that PL/M is a stripped back PL/I. Having never looked at any PL/I source code until today I can see a vague passing resemblance but that might be the all-caps text fooling me.

I never used PL/M myself but have been looking at it closely recently, it was - at least initially - closely allied with the 8080, the language itself had facilities for interacting with DMA devices too it seems, may I ask, what kind of work did you do with PL/M?

PL/I was a designed, specified programming language. It was created after a detailed analysis of assembler, Cobol, Algol and Fortran. It had rich bit and packing/alignment features due to the cost of memory. Many programmers back then (1960s) had solid assembler experience and were no strangers to packing data into fields to conserve memory, so it was high on the list of the designers to give the PL/I programmer a similar ability to work at that level, this are received a lot of attention - all these years later that same mindset could apply to resource constrained MCUs.

It was a huge success and saw huge uptake in UK and Europe as well as Eastern block countries, it was marketed there differently to the US, where the costs of the language were rather high and this perhaps the biggest reason it became less well known over here. My first real programming job was a trainee programmer on PL/I on IBM mainframes in Enfield, London. Later I was hired to work in the City of London because Stratus hardware fault tolerant computer had started to see like hot cakes in mission critical environments (stock exchanges, investment banks, market makers). Stratus was interesting and had an OS written in PL/I subset G and was the preferred language to use on these machines, PL/I programmers were thus in demand and I got a job on that basis. The CTO of Stratus was interesting too, had a business developing and selling compilers (including PL/I) for Burroughs, DEC, Data General, Prime etc.

For those interested in computer history here's an interview with him (https://archive.computerhistory.org/resources/access/text/2013/03/102743157-05-01-acc.pdf).

I've been particularly concerned recently with the grammar for a new language, once a grammar is defined there's no going back, one must live with the consequences of the choice for the lifetime of the language, PL/I's grammar is not perfect but it is perhaps one of the best imperative language grammars around IMHO.

I also developed a Subset G PL/I compiler during the 90s so I have solid experience of this, that compiler was never strictly completed but did generate linkable runnable COFF object modules for Windows NT, I wrote the entire compiler (except the small bootstrap start code which was assembler) in C, initially Borland then later Visual C, writing a compiler for PL/I in C was a unique learning experience, I saw the good. the bad and the ugly of both languages, up close, this was all before the internet too, so all research and design was driven by books and histiorical reprints of articles and papers from my local library.


Title: Re: A microcontroller programming language
Post by: rstofer on December 06, 2022, 03:11:58 pm
PL/M source code here:

http://www.cpm.z80.de/source.html (http://www.cpm.z80.de/source.html)

It is written in Fortran 66.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 06, 2022, 03:27:34 pm
PL/M source code here:

http://www.cpm.z80.de/source.html (http://www.cpm.z80.de/source.html)

It is written in Fortran 66.

That's pretty interesting, I've never see Fortran up close, also the PL/M to C translators written in C is intriguing too, thanks for posting that.
Title: Re: A new, hardware "oriented" programming language
Post by: Brianf on December 06, 2022, 03:42:09 pm
What was PL/M like, and should it come back?  I've never used it.

As it looks like I'm the only one here who has used PL/M (for about 3 years) I'd put is somewhere between C and assembler. I still have a bit of a soft spot for it and would happily use it if a compiler were available.

I read above that PL/M is a stripped back PL/I. Having never looked at any PL/I source code until today I can see a vague passing resemblance but that might be the all-caps text fooling me.

I never used PL/M myself but have been looking at it closely recently, it was - at least initially - closely allied with the 8080, the language itself had facilities for interacting with DMA devices too it seems, may I ask, what kind of work did you do with PL/M?

We were doing remote control and monitoring of industrial equipment.

8085 processor, 8259 interrupt controllers, 8237 DMA controllers, 8251 UARTs, 8274 HDLC/SDLC controllers, plus RAM and EPROM on the processor board. The 8251s talked in/out to multiple serial ports. The 8274 implemented an 800kbps interboard comms link. All running under DMA to reduce CPU usage.

8031 processor (don't ask) and a TMS9918 (?) VDP on the graphics board, commands via a 8274 on the interboard comms link. Also using PL/M (PL/M-51).

8751 (old top, UV window) in the remote units with analogue inputs, digital outputs via relays, and digital inputs.
Title: Re: A new, hardware "oriented" programming language
Post by: Sherlock Holmes on December 06, 2022, 05:09:35 pm
What was PL/M like, and should it come back?  I've never used it.

As it looks like I'm the only one here who has used PL/M (for about 3 years) I'd put is somewhere between C and assembler. I still have a bit of a soft spot for it and would happily use it if a compiler were available.

I read above that PL/M is a stripped back PL/I. Having never looked at any PL/I source code until today I can see a vague passing resemblance but that might be the all-caps text fooling me.

I never used PL/M myself but have been looking at it closely recently, it was - at least initially - closely allied with the 8080, the language itself had facilities for interacting with DMA devices too it seems, may I ask, what kind of work did you do with PL/M?

We were doing remote control and monitoring of industrial equipment.

8085 processor, 8259 interrupt controllers, 8237 DMA controllers, 8251 UARTs, 8274 HDLC/SDLC controllers, plus RAM and EPROM on the processor board. The 8251s talked in/out to multiple serial ports. The 8274 implemented an 800kbps interboard comms link. All running under DMA to reduce CPU usage.

8031 processor (don't ask) and a TMS9918 (?) VDP on the graphics board, commands via a 8274 on the interboard comms link. Also using PL/M (PL/M-51).

8751 (old top, UV window) in the remote units with analogue inputs, digital outputs via relays, and digital inputs.

Wow, OK so hardly some trivial system, I found the exact manual here (http://www.bitsavers.org/pdf/intel/ISIS_II/121966-003_PLM-51_Users_Guide_Jan84.pdf).

Seems PL/M (unlike PL/I) does reserve its keywords.


Title: Re: A microcontroller programming language
Post by: Brianf on December 06, 2022, 06:49:12 pm
And then there's the book...

https://www.amazon.co.uk/Guide-P-Programming-Microcomputer-Applications/dp/0201045753/ (https://www.amazon.co.uk/Guide-P-Programming-Microcomputer-Applications/dp/0201045753/)
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 06, 2022, 07:19:19 pm
The fun part is that the thread is starting to look like a popularity contest (which is what ends up happening in many programming language threads anyway) rather than stick to purely technical points. Which usually means game over, but I don't wanna ruin the thread. Good luck!

It seems you are a French speaker, so on that basis let me ask you a question, how would you update this JSON's "fr" entry to use the most appropriate French terms for the various keywords:

Quote
{
  "en": {
    "IsNotAKeyword": "0",
    "Argument": "argument",
    "Binary": "binary",
    "Boolean": "boolean",
    "Builtin": "builtin",
    "By": "by",
    "Call": "call",
    "Coroutine": "coroutine",
    "Decimal": "decimal",
    "Declare": "declare",
    "End": "end",
    "Enum": "enum",
    "Fixed": "fixed",
    "Float": "float",
    "Function": "function",
    "Goto": "goto",
    "If": "if",
    "Internal": "internal",
    "Interrupt": "interrupt",
    "Language": "lingua",
    "Loop": "loop",
    "Namespace": "namespace",
    "Out": "out",
    "Private": "private",
    "Procedure": "procedure",
    "Public": "public",
    "Readonly": "readonly",
    "Ref": "ref",
    "Return": "return",
    "Returnon": "returnon",
    "Singlet": "singlet",
    "Static": "static",
    "String": "string",
    "Structure": "structure",
    "To": "to",
    "Type": "type",
    "Until": "until",
    "Using": "using",
    "While": "while",
    "Varying": "varying",
    "Yield": "yield",
    "Arg": "arg",
    "Bin": "bin",
    "Dcl": "dcl",
    "Dec": "dec",
    "Func": "func",
    "Proc": "proc",
    "Struct": "struct"
  },

  "fr": {
    "IsNotAKeyword": "0",
    "Argument": "argument",
    "Binary": "binaire",
    "Boolean": "booléenne",
    "Builtin": "intégré",
    "By": "by",
    "Call": "appeler",
    "Coroutine": "coroutine",
    "Decimal": "décimal",
    "Declare": "déclarer",
    "End": "fin",
    "Enum": "enum",
    "Fixed": "fixe",
    "Float": "flottant",
    "Function": "fonction",
    "Goto": "goto",
    "If": "si",
    "Internal": "interne",
    "Interrupt": "interrompre",
    "Language": "lingua",
    "Loop": "boucle",
    "Namespace": "namespace",
    "Out": "depuis",
    "Private": "privé",
    "Procedure": "procédé",
    "Public": "public",
    "Readonly": "readonly",
    "Ref": "ref",
    "Return": "retour",
    "Returnon": "retour si",
    "Singlet": "singlet",
    "Static": "static",
    "String": "chaîne",
    "Structure": "structure",
    "To": "to",
    "Type": "type",
    "Until": "avant",
    "Using": "using",
    "While": "tandis que",
    "Varying": "varying",
    "Yield": "donner",
    "Arg": "arg",
    "Bin": "bin",
    "Dcl": "dcl",
    "Dec": "déc",
    "Func": "fonc",
    "Proc": "proc",
    "Struct": "struct"
  }

The current terms are very crude, looked up hastily and no doubt poorly, my Wife speaks French like a native (she studied linguistics and majored in French) but she's not familiar with technology or the intent of these various keywords.

The lexical analyzer and parser has no problem handling keywords with multiple terms either like "go to" (EN, Goto) or "tandis que" (FR, While) or "retour si" (FR, ReturnOn) etc, accent characters are fine too.
Title: Re: A new, hardware "oriented" programming language
Post by: tggzzz on December 06, 2022, 07:53:46 pm
What was PL/M like, and should it come back?  I've never used it.

As it looks like I'm the only one here who has used PL/M (for about 3 years) I'd put is somewhere between C and assembler. I still have a bit of a soft spot for it and would happily use it if a compiler were available.

I read above that PL/M is a stripped back PL/I. Having never looked at any PL/I source code until today I can see a vague passing resemblance but that might be the all-caps text fooling me.

I never used PL/M myself but have been looking at it closely recently, it was - at least initially - closely allied with the 8080, the language itself had facilities for interacting with DMA devices too it seems, may I ask, what kind of work did you do with PL/M?

PL/I was a designed, specified programming language. It was created after a detailed analysis of assembler, Cobol, Algol and Fortran. It had rich bit and packing/alignment features due to the cost of memory. Many programmers back then (1960s) had solid assembler experience and were no strangers to packing data into fields to conserve memory, so it was high on the list of the designers to give the PL/I programmer a similar ability to work at that level, this are received a lot of attention - all these years later that same mindset could apply to resource constrained MCUs.

It was a huge success and saw huge uptake in UK and Europe as well as Eastern block countries, it was marketed there differently to the US, where the costs of the language were rather high and this perhaps the biggest reason it became less well known over here. My first real programming job was a trainee programmer on PL/I on IBM mainframes in Enfield, London. Later I was hired to work in the City of London because Stratus hardware fault tolerant computer had started to see like hot cakes in mission critical environments (stock exchanges, investment banks, market makers). Stratus was interesting and had an OS written in PL/I subset G and was the preferred language to use on these machines, PL/I programmers were thus in demand and I got a job on that basis. The CTO of Stratus was interesting too, had a business developing and selling compilers (including PL/I) for Burroughs, DEC, Data General, Prime etc.

For those interested in computer history here's an interview with him (https://archive.computerhistory.org/resources/access/text/2013/03/102743157-05-01-acc.pdf).

I've been particularly concerned recently with the grammar for a new language, once a grammar is defined there's no going back, one must live with the consequences of the choice for the lifetime of the language, PL/I's grammar is not perfect but it is perhaps one of the best imperative language grammars around IMHO.

I also developed a Subset G PL/I compiler during the 90s so I have solid experience of this, that compiler was never strictly completed but did generate linkable runnable COFF object modules for Windows NT, I wrote the entire compiler (except the small bootstrap start code which was assembler) in C, initially Borland then later Visual C, writing a compiler for PL/I in C was a unique learning experience, I saw the good. the bad and the ugly of both languages, up close, this was all before the internet too, so all research and design was driven by books and histiorical reprints of articles and papers from my local library.

If you are interested in PL/M, you ought to understand Coral-66 (that's 1966). Key points emphasised...

Quote
Coral 66 is a general-purpose programming language based on ALGOL 60, with some features from Coral 64, JOVIAL, and FORTRAN. It includes structured record types (as in Pascal) and supports the packing of data into limited storage (also as in Pascal). Like Edinburgh IMP it allows embedded assembler, and also offers good run-time checking and diagnostics. It is specifically intended for real-time and embedded applications and for use on computers with limited processing power, including those limited to fixed point arithmetic and those without support for dynamic storage allocation.
The language was an inter-service standard for British military programming, and was also widely adopted for civil purposes in the British control and automation industry. It was used to write software for both the Ferranti [2] and GEC computers from 1971 onwards. Implementations also exist for the Interdata 8/32, PDP-11, VAX, Alpha platforms and HP Integrity servers; for the Honeywell, and for the Computer Technology Limited (CTL, later ITL) Modular-1; [3] as well as for SPARC running Solaris and Intel running Linux.
A variant of Coral 66 was developed during the late 1970s/early 1980s by the British GPO, in conjunction with GEC, STC and Plessey, for use on the System X digital telephone exchange control computers, known as PO-CORAL. This was later renamed BT-CORAL when British Telecom was spun off from the Post Office. Unique features of this language were the focus on real-time execution, message processing, limits on statement execution between waiting for input, and a prohibition on recursion to remove the need for a stack.
As Coral was aimed at a variety of real-time work, rather than general office DP, there was no standardised equivalent to a stdio library. IECCA recommended a primitive I/O package to accompany any compiler (in a document titled Input/Output of Character data in Coral 66 Utility Programs). Most implementers avoided this by producing Coral interfaces to existing Fortran and, later, C libraries.
Perhaps CORAL's most significant contribution to computing was the enforcement of quality control in commercial compilers. [3] To have a CORAL compiler approved by IECCA, and thus allowing a compiler to be marketed as a CORAL 66 compiler, the candidate compiler had to compile and execute an official suite of 25 test programs and 6 benchmark programs. The process was part of the BS 5905 approval process. This methodology was observed and adapted later by the United States Department of Defense for the official certification of Ada compilers.

IIRC it included a fixed point data type, with associated arithmetic operations.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 06, 2022, 08:56:46 pm
So lets talk about structures.

Several important aspects to this area are:

1. Runtime ordering of member fields
2. Alignment of member fields
3. Padding between member fields
4. Packing

The runtime ordering (that is order of fields when ordered by offset from start of structure) is fixed in C and C++, one must manually adjust the lexical order to effect a change to the runtime order.

So this suggest something like:

Code: [Select]

def header struct order (auto)
   ...
end

def header struct order (explicit)
   ...
end


With some default when "order" is omitted. The "auto" meaning to let the compiler order the members my decreasing alignment needs.

Alignments could be specified at the structure level, or perhaps at the member level too.

Padding is a function of alignment I suppose, so is implicit or could be instead of:

Code: [Select]

def header struct
   length bin(16)
   code byte(1)
   padding(1)
   type bin(32)
end

or

Code: [Select]
def header struct aligned
   length bin(16)
   code byte(1)
   type bin(16)
end


Specifying "aligned" means you cant also specify any padding and vice versa. The "padding" keyword can appear many times inside a structure definition and basically forces the offset of following fields, so "padding" would prevent "order(auto)" since it's antagonistic to that.


One could also have "unused(n)" as well as "padding(n)". In that case an "unused" allocates real space but it can't be accessed, if it is, the compiler can report an error. That means we can cater for fields that are present and might be aligned poorly, but we don't care because we will never access that field.

One should be able to devise a set of keywords and options that give huge control over all aspects of structure's memory layout like this, meaning one can meet any conceivable need.

Thoughts?
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 06, 2022, 11:24:29 pm
Alignment and packing can be defined in many programming languages these days. Even C. This is certainly useful but nothing new. Ada is by far the most flexible for this.
Automatic ordering of members is rarer - well, some languages do not define a specific order for "structures", so compilers are allowed to reorder the members as they see fit. Not particularly new either, but indeed not in C or C++. While not mind-blowingly useful, that would give opportunities for optimizations when the developer doesn't care about order, which is relatively frequent indeed unless you map structs to low-level objects.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 06, 2022, 11:50:06 pm
Alignment and packing can be defined in many programming languages these days. Even C. This is certainly useful but nothing new. Ada is by far the most flexible for this.
Automatic ordering of members is rarer - well, some languages do not define a specific order for "structures", so compilers are allowed to reorder the members as they see fit. Not particularly new either, but indeed not in C or C++. While not mind-blowingly useful, that would give opportunities for optimizations when the developer doesn't care about order, which is relatively frequent indeed unless you map structs to low-level objects.


Yes, I know this is nothing new, the language is new but not the underlying concepts.

I'm leaning toward this now as the way to declare structures/types and explicitly include padding and unused regions:

Code: [Select]

def message struct
    sequence bin(31)
    tag      byte(2)

    // Note all members that are padding/ignored must have same name within the struct

    XXXX    byte (5) padding
    XXXX    byte (3) ignored // ignore the 3 byte checksum
end


This seems neat to me, how many times have we had to declare unused fields in complex structures and been forced to name them "ignore_1" and "ignore_2" and so on? I have and it's these kind of things that interest me with a new language.

So here you can use any name you want for a "padding" or "ignored" field but with the proviso that every such field within a struct/type must use the very same name, look at the example, I chose XXXX and look how easy it is to visually scan and see that name repeated, it's clear they are either ignored or padding, very helpful when scanning visually over potentially complex code...

This is trivial for a good parser and also very rational, since you can't ever refer to an ignored field or a padding field, there's no reason for them not to have the same name! The name is only a means to and end anyway, a means by which the code generation phase can determine the address being referred to.



Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 13, 2022, 04:24:31 am
The fun part is that the thread is starting to look like a popularity contest (which is what ends up happening in many programming language threads anyway) rather than stick to purely technical points. Which usually means game over, but I don't wanna ruin the thread. Good luck!

It seems you are a French speaker, so on that basis let me ask you a question, how would you update this JSON's "fr" entry to use the most appropriate French terms for the various keywords:
(...)

I missed this thread which is why I didn't answer. Note that I don't think translating programming language keywords is ever a good idea (it has been done before, and it eventually died.)
But anyway, I'll have a look at the list if I get the time.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 13, 2022, 06:12:37 pm
The fun part is that the thread is starting to look like a popularity contest (which is what ends up happening in many programming language threads anyway) rather than stick to purely technical points. Which usually means game over, but I don't wanna ruin the thread. Good luck!

It seems you are a French speaker, so on that basis let me ask you a question, how would you update this JSON's "fr" entry to use the most appropriate French terms for the various keywords:
(...)

I missed this thread which is why I didn't answer. Note that I don't think translating programming language keywords is ever a good idea (it has been done before, and it eventually died.)
But anyway, I'll have a look at the list if I get the time.

Very well, but to infer it is never a good idea is something of a stretch surely? The case here though is that the ability to freely and reliably support keywords from other languages is an unanticipated consequence of the grammar's lack of reserved words. Because of that it is almost a no brainer so is something I'd leverage as low hanging fruit.

All we have to do is change every keyword to say French or Dutch and then add/change the language directive and we're done. I already played around with this and the language code (like "en" or "fr") is now an argument to the lexical analyzer.

Even better, one could alter the language instantly within an IDE or editor without to much effort, a syntax aware editor could do that, just parse the code, and replace every keyword in the parse tree with its French, German etc, equivalent and you're done.

I've been looking at Antlr4 as a possible means to more formally develop the grammar but I don't yet know if it is capable of handling some of the grammar needs, I'm chatting to Antlr people to try and get clarity.

Anyway, thanks for taking a look.





Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 13, 2022, 06:21:49 pm
I've been doing more investigation into features that might be helpful with a new language. One of these (again) comes right from PL/I.

That language supports "pointer" as a data type but it also support another related data type "offset".

That's right, you can declare structure members as "offset" and that means the runtime address of the datum is relative to some base address. This means one can create fully relocatable linked data structures, so long as the base address is established the data structure "works" fine.

This is closely associated with PL/I's "area" support, effectively user declared heaps. You can declare a "heap" and allocate and free within that heap, then later you can (if you wanted to) copy the area or persist and so on, and later reload the area into memory and the data structures in there, linked lists, two way lists, trees etc will function perfectly (if offsets are used rather than pointers).

Anyway these are ideas that might have some utility in an MCU setting.
Title: Re: A microcontroller programming language
Post by: DavidAlfa on December 13, 2022, 11:51:21 pm
(https://media.tenor.com/R74sMGxfIK4AAAAC/homer-simpson-i-see.gif)
Title: Re: A microcontroller programming language
Post by: MIS42N on December 14, 2022, 05:24:10 am
    No reserved words, thus enabling new keywords to be added over time.
NO - learning a language has a lot to do with knowing the reserved words and what they do. How do you give a program to another person to maintain if you've cooked up your own set of names. If a future conflict with a new reserved word, that's what global find and replace does well. I quite often write code with labels like X57 then come back and do a replace later with something more meaningful. It saves time if labels are short to type. A global search and replace finds labels that have no other reference, so asks the question, did I forget something.

    Support 'bit' as a native data type.
YES - I think COBOL has had this for quite a while with USAGE BIT and PIC 99 COMP etc. By specifying exactly what you want, it's up to the compiler to figure it out - big endian, little endian etc.

    Support 'strings' as a native type, BCD/decimal as well.
YES - as above

    Support for namespaces.
Some sort of name scope is useful.

    Computed gotos.
YES

    Flexible alignment, packing and padding directives.
? - maybe. Shouldn't the compiler handle this anyway, that's what they are there for.

    Nested functions.
This is for small microcontrollers? could get messy.

    Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.
    Support for an async/await model.
I would prefer not to hand these to a compiler.

I tried to come to grips with the PIC XC8 compiler but getting it to play nicely with my assembler was exhausting and the code was incomprehensible. So back to all assembler. A good library of assembler sources that can be cut and pasted are very useful. Add plenty of documentation, I find it easier than just about any other language that isn't well documented. And I know exactly what is going on. Using a higher level language rarely produces optimal code. Perhaps one day an AI can be trained by reading a couple of million lines of C (or whatever) code and the hand coded assembler to do the same thing. Then I might use it.

Regarding which, the newer PIC16 architecture fixes many of the problems with the older architecture. It now has arithmetic with carry (which just about every other processor had for ever). Two 16 bit index registers that can access data and program memory with pre/post indexing or offsets. Data memory can be accessed via the page method or as a linear space. Interrupts save (and RETFIE restores) the most significant registers (8 bytes) in a single instruction cycle. And part of addressable 'data' memory gives access to the call/return stack and the saved registers in an interrupt. Some of the peripherals are enhanced too - timer 1 and the PWM can use the base clock (up to 64MHz) instead of the instruction clock (base/4).

Somewhat off topic.
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 14, 2022, 05:53:01 am
I would make the following changes: (only kept what needs changed here)

  "fr": {
   
    "Boolean": "booléen", -- more neutral form, 'booléenne' is the feminine version, although feminazis may not like the neutral form, which is masculine

    "Builtin": "intégré", -- this one doesn't sound quite right to me (even if the general meaning is correct), but I don't really know how to translate it better without more context about what 'builtin' would mean in said prog. language

    "By": "par",

    "Goto": "aller à/en",

    "Interrupt": "interruption",

    "Language": "langage",

    "Namespace": "espace de noms", -- which is correct but frankly looks odd and overly verbose

    "Out": "sortie",

    "Procedure": "procédure",

    "Readonly": "lecture seule",

    "Singlet": "singleton", -- if by 'singlet' you mean a set or list of 1 element

    "Static": "statique",

    "To": "à",

    "Until": "jusqu'à ce que",

    "Using": "utilisant",

    "While": "tant que",

    "Varying": "variant",

    "Yield": "céder",

  }

Some keywords are clunky to translate, and I don't think it's a good idea as I said (and MIS42N said as well.) But there you go.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 14, 2022, 12:25:05 pm
I would make the following changes: (only kept what needs changed here)

  "fr": {
   
    "Boolean": "booléen", -- more neutral form, 'booléenne' is the feminine version, although feminazis may not like the neutral form, which is masculine

    "Builtin": "intégré", -- this one doesn't sound quite right to me (even if the general meaning is correct), but I don't really know how to translate it better without more context about what 'builtin' would mean in said prog. language

    "By": "par",

    "Goto": "aller à/en",

    "Interrupt": "interruption",

    "Language": "langage",

    "Namespace": "espace de noms", -- which is correct but frankly looks odd and overly verbose

    "Out": "sortie",

    "Procedure": "procédure",

    "Readonly": "lecture seule",

    "Singlet": "singleton", -- if by 'singlet' you mean a set or list of 1 element

    "Static": "statique",

    "To": "à",

    "Until": "jusqu'à ce que",

    "Using": "utilisant",

    "While": "tant que",

    "Varying": "variant",

    "Yield": "céder",

  }

Some keywords are clunky to translate, and I don't think it's a good idea as I said (and MIS42N said as well.) But there you go.

Much appreciated. The language support is easy to implement as an option, like a "pragma" in C. Not used unless someone wants it. English will be default. There's little cost in doing it, low hanging fruit.
Title: Re: A microcontroller programming language
Post by: JPortici on December 14, 2022, 12:37:43 pm
Do you know how much i hate that i have to translate excel/calc functions because one colleague's pc is set to italian locale and my pc is set to english locale?
Translatable keywords is as bas as no reserved words

And there is nothing bad in having reserved words, even if they are not in your native language. I remember it took me a time to understand why word "x" was chosen. After all in middle school we were learning programming alongside english. But it didn't really matter. I only knew i had to follow the syntax
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 14, 2022, 01:09:23 pm
Do you know how much i hate that i have to translate excel/calc functions because one colleague's pc is set to italian locale and my pc is set to english locale?
Translatable keywords is as bas as no reserved words

And there is nothing bad in having reserved words, even if they are not in your native language. I remember it took me a time to understand why word "x" was chosen. After all in middle school we were learning programming alongside english. But it didn't really matter. I only knew i had to follow the syntax

Say you use a new language, develop some significant software, important to you, your company. Assume the version 1.0 of the language you used had no keyword "async". Assume further, one of the developers liked the word and used it for say a function name, all good.

Now back to me, the language designer working on v 2.0, in this new version we are seeking to add a new keyword to represent asynchronous IO and choose the keyword "async" what will happen when we compile your code, using v2.0 (where "async" is now a new reserved word)?
Title: Re: A microcontroller programming language
Post by: JPortici on December 14, 2022, 03:49:50 pm
if you have existing code base you have to mantain, you don't have to suddntly use v2.0 in that project if you don't need features from 2.0, there are many compilers that give an option to use a certain version of the language
Title: Re: A microcontroller programming language
Post by: JPortici on December 14, 2022, 05:46:05 pm
Thinking about it.. You know one feature that C doesn't really have but that i would like to have in a language?
I would like to have such options to be present in a source file:
The reason is, let's say i want this particular code file to compile with a specific version of the language, or with this specific option
so that i don't have to remember to do it on every project
Also would be easier to document

I think in C compilers we can do some on a per function basis, but not on the file scope
Title: Re: A microcontroller programming language
Post by: cfbsoftware on December 14, 2022, 10:52:35 pm
Now back to me, the language designer working on v 2.0, in this new version we are seeking to add a new keyword to represent asynchronous IO and choose the keyword "async" what will happen when we compile your code, using v2.0 (where "async" is now a new reserved word)?
The most common initial complaint I hear from newcomers to languages like Oberon and Modula-2 are their use of case-sensitive identifiers and all-uppercase reserved words.

However, a significant benefit of these features are that name clashes of this sort never happen - unless you are in the habit of using all-uppercase identifiers in your own code. You can also freely use lowercase or mixed-case identifiers in your own code without ever having to think about existing reserved words.
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 15, 2022, 12:18:30 am
Now back to me, the language designer working on v 2.0, in this new version we are seeking to add a new keyword to represent asynchronous IO and choose the keyword "async" what will happen when we compile your code, using v2.0 (where "async" is now a new reserved word)?
The most common initial complaint I hear from newcomers to languages like Oberon and Modula-2 are their use of case-sensitive identifiers and all-uppercase reserved words.

However, a significant benefit of these features are that name clashes of this sort never happen - unless you are in the habit of using all-uppercase identifiers in your own code. You can also freely use lowercase or mixed-case identifiers in your own code without ever having to think about existing reserved words.

While I don't like those all uppercase keywords all that much, you make a good point.
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 15, 2022, 11:08:29 am
if you have existing code base you have to mantain, you don't have to suddntly use v2.0 in that project if you don't need features from 2.0, there are many compilers that give an option to use a certain version of the language

People miss this all the time when complaining about trivial namespace clashes. Changing to a new language version is like changing to a different language, just smaller change, but it doesn't just happen. So geez, just keep using the language version you are using, nothing suddenly breaks. If you truly find a new language version to provide some significant improvement to warrant an update of the project to new language version (1% likely), and if it happens some new keyword clashes with a variable name in your project (0.1% likely; total 0.001%), then rename the damn variable (usually 5-minute job, but in 1% cases (approaching 0.00001% total), maybe there is more to it and it takes a full day).

Ecosystem is important. The good feature in C ecosystem is, language versions are actually standardized, and compilers target specific versions (usually chosen by a command line flag). It's not a mess like python where you have two totally incompatible versions and some magical "python" symlink where you don't know if it launches python2 or python3.

As always, made-up solutions to made-up "problems". Look at C, C does this right, as usual, as much as people choose to hate it. (And no, this still is not a personal attack).
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 15, 2022, 11:13:10 am
Thinking about it.. You know one feature that C doesn't really have but that i would like to have in a language?
I would like to have such options to be present in a source file:
The reason is, let's say i want this particular code file to compile with a specific version of the language, or with this specific option
so that i don't have to remember to do it on every project
Also would be easier to document

Seems nice on surface, but it's a good idea to write a makefile and make that part of your process, i.e., put it in revision control system and mentally treat it as something equally important to all those source files.

You can totally generate different rules (command line options) in makefile on a source file basis. The good side compared to your suggestion is, you don't have to copy-paste the settings to each source file and hope they are kept in sync, because in realistic projects, 99% of files use the same settings 99% of the time, so good to have in one file (makefile).

If you have difficulty remembering the "good" settings, just copy the whole makefile from the previous project - or make yourself a good template. This is what I do.
Title: Re: A microcontroller programming language
Post by: JPortici on December 15, 2022, 12:35:44 pm
Thinking about it.. You know one feature that C doesn't really have but that i would like to have in a language?
I would like to have such options to be present in a source file:
The reason is, let's say i want this particular code file to compile with a specific version of the language, or with this specific option
so that i don't have to remember to do it on every project
Also would be easier to document

Seems nice on surface, but it's a good idea to write a makefile and make that part of your process, i.e., put it in revision control system and mentally treat it as something equally important to all those source files.

You can totally generate different rules (command line options) in makefile on a source file basis. The good side compared to your suggestion is, you don't have to copy-paste the settings to each source file and hope they are kept in sync, because in realistic projects, 99% of files use the same settings 99% of the time, so good to have in one file (makefile).

If you have difficulty remembering the "good" settings, just copy the whole makefile from the previous project - or make yourself a good template. This is what I do.

but then i have to remember to do it every time i use that source file. The point is to not having to remember stuff
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 15, 2022, 12:40:01 pm
If you have difficulty remembering the "good" settings, just copy the whole makefile from the previous project - or make yourself a good template. This is what I do.

but then i have to remember to do it every time i use that source file. The point is to not having to remember stuff

You lost me. Remember to do what exactly? Copying one makefile is too much to remember?

I think the opposite is true. If the compiler options, language standards etc. are specified in source files directly, that's a lot to remember, on every file! I rather copy one file than copy-paste 10 lines of configuration in each project source file.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 15, 2022, 01:57:15 pm
but then i have to remember to do it every time i use that source file. The point is to not having to remember stuff
I keep stuff in units; currently one directory per unit test/experiment/detail.  If a file requires special compiler options, or I mix C and C++ (for example because g++ still doesn't support named address spaces, unlike e.g. clang), I document it.  I have a short comment in the Makefile, plus a README in that directory.

My brain capacity is so limited I couldn't remember all the details even if I tried.  Documentation is the tool I use to not be limited by my poor memorization capabilities.  In the end, this has taught me to read documentation and source quite quickly, to find salient points, which helps in many things from debugging to adapting existing code for new uses.  I honestly do not believe for a second that not being able to memorize stuff is a handicap; I actually believe the opposite (never relying on memorizing stuff) is more efficient.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 15, 2022, 02:10:55 pm
I keep stuff in units; currently one directory per unit test/experiment/detail.  If a file requires special compiler options, or I mix C and C++ [..] I document it.  I have a short comment in the Makefile, plus a README in that directory.

I do exactly the same here  :D
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 15, 2022, 03:41:40 pm
Well I'm encouraged by some recent developments. First the lead engineer on an unrelated foreign language programming system has been helpful and we are in communication.

Second, I have been evaluating Antlr as a possible way to create a parser for a new language, having no reserved words seemed like a stumbling block for Antlr and I thought I'd have to hand craft a parser again, doable but a lot of work.

Well Antlr came through, after a few days experimenting I got what I need, this is not the new language, but a crude test of the grammar that I used with Antlr, this supports keywords as identifiers:

Code: [Select]
call = 1234
goto done
ball = 1234
proc revalidate() { }
call ball()
goto = 999
procedure revalidate() { }
proc = 999;


[attachimg=1]

This is extremely encouraging the effort saved by using Antlr will be huge, seriously huge.
Title: Re: A microcontroller programming language
Post by: JPortici on December 15, 2022, 04:23:16 pm
I meant: just as when using certain compilers you need to add
#pragma pack 1
before defining packed structures, you could add something like
#pragma stdandard C90
so that the whole file will be compiled using C90 rules

one advantage could be that the information is within the source file instead of being in another file in the directory, which may be overlooked when copying/importing
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 15, 2022, 05:02:03 pm
Better examples:

Code: [Select]
call = 1234;

goto done;

ball = 1234;

proc revalidate()
{
;
}
call ball();
goto = 999;
procedure revalidate()
{

}
proc = 999;
ptr->xyz = 2;
proc call ()
{
call goto();

proc inner()
{
do_stuff = 0;
}
}

Parse tree is:

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 15, 2022, 05:14:12 pm
Parse tree[attachimg=1]
Title: Re: A microcontroller programming language
Post by: tggzzz on December 15, 2022, 06:43:34 pm
If you have difficulty remembering the "good" settings, just copy the whole makefile from the previous project - or make yourself a good template. This is what I do.

but then i have to remember to do it every time i use that source file. The point is to not having to remember stuff

You lost me. Remember to do what exactly? Copying one makefile is too much to remember?

I think the opposite is true. If the compiler options, language standards etc. are specified in source files directly, that's a lot to remember, on every file! I rather copy one file than copy-paste 10 lines of configuration in each project source file.

If it was copying one file, you might have a point. But since every project has different source files/directories/libraries, and the compiler version has changed, that isn't the case.

Normally you will need to extract the command line arguments for the components that are also in the new project, and check to see that "improved" compiler optimisations don't require extra or changed flags. Not trivial, and there is a distinct possibility of error.

Putting comments in the source files isn't a panacea, either.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 15, 2022, 07:33:33 pm
pragma is always bad
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 15, 2022, 07:48:13 pm
Ecosystem is important. The good feature in C ecosystem is, language versions are actually standardized, and compilers target specific versions (usually chosen by a command line flag).

Not only that, but apart from a very few (that the committee tries to minimize) points, backwards compatibility is largely maintained, which is also a very strong point and makes switching to a newer revision of the standard pretty easy. While breaking backwards compatibility is very common in newer languages.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 16, 2022, 06:54:31 pm
The grammar is coming along well, Antlr is very impressive, this parses absolutely fine:

Code: [Select]
counter = 100;

/* declare some variables named the same as keywords */

dcl dcl dec based(counter);  /* dcl is a keyword ! */

dcl counter binary;

declare goto binary; /* goto is a keyword ! */



goto call; /* call is a keyword ! */



call declare(); /* declare is a keyword */

call declare (1,2,3);

return (123);

if if = then then
   dcl realm binary;
   call reboot(0);
   go to end;
end;


Yielding this tree:

[attachimg=1]

Note the two forms of goto (PL/I always supported these two forms) cause no issues, not a huge advocate of this but the fact this can be reliably parsed will help with foreign languages that might use multiple terms to express certain actions.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 16, 2022, 07:41:12 pm
I can feel the power...

Code: [Select]
if if = then then
   dcl realm binary;
   call reboot(0);
   go to end;
else
   return (123);
   if = 44;
   end = then + else;
end;

[attachimg=1]
Title: Re: A microcontroller programming language
Post by: rstofer on December 16, 2022, 07:55:28 pm
If I employed somebody who wrote code like that they would be out of a job the very first day.

Just because you can is not a reason for actually doing.

I am not convinced of the absence of keywords being a good idea.
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 16, 2022, 08:41:37 pm
thats called reinventing the wheel the wrong way ;D hence my 1st post in this thread still applies, but i would like to be proven wrong though... but if you want to be in the list, be my guest... https://www.hongkiat.com/blog/bizarre-insane-programming-languages/ (https://www.hongkiat.com/blog/bizarre-insane-programming-languages/)
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 16, 2022, 11:11:04 pm
I can feel the power...

Code: [Select]
if if = then then
   dcl realm binary;
   call reboot(0);
   go to end;
else
   return (123);
   if = 44;
   end = then + else;
end;

(Attachment Link)

There are various reserved words in the C language, such as 'if'.  I suspect, it is because it is known and learnt from experience, that if you don't have enough (of the right kind of) reserved words.  It can lead to too many ways in which programmers can accidentally introduce faults in the source code.  Which (due to the lack of reserved words), can make the compiler, incorrectly ignore without generating any warnings or error messages, mistake(s) in the source code.

Also it can lead to valid source code, which is especially confusing for the unwary programmer.

C is not the best of examples, for my recent posts in this thread.  Because its #define concept, relies on programmer discipline, to not inappropriate use it, to make significantly poor (judgement) quality source code, which is for example, very difficult to understand by anyone, other than the original programmer.

As soon as you commit to distinguishing a keyword from an identifier on the basis of its spelling, rather than its context you pay a price. That price is the inability to freely add new keywords over time.

I do not advocate intentionally naming things the same as a keyword, but that's a style, coding standards issue not a grammar issue.

As many have said already in this thread, a developer should know and understand the programming language they use, not become "confused" over something so basic.

Regarding C, the keywords are not reserved to prevent their use as identifiers but for ease of parsing, also C could never do otherwise given that it (inexcusably) represents (void) function invocation with the name of the function alone and not a keyword, this was a mistake and you and others are still paying the price.

There's nothing complicated about incorporating style diagnostics, polite reminders like "The identifier 'yield' is also a language keyword, consider renaming this for improved readability".

Finally when it comes to enabling confusing looking code, C takes the cake:

Code: [Select]

void f(char *x, char*y, int z)
{
    int z2 = (z + 7) / 8;
    switch (z % 8)
    {
        case 0:
        do
        {
            *x++ = *y++;
        case 7:
            *x++ = *y++;
        case 6:
            *x++ = *y++;
        case 5:
            *x++ = *y++;
        case 4:
            *x++ = *y++;
        case 3:
            *x++ = *y++;
        case 2:
            *x++ = *y++;
        case 1:
            *x++ = *y++;
            } while (--z2 > 0);
    }
}


https://www.youtube.com/watch?v=8yfcwY2xLGE (https://www.youtube.com/watch?v=8yfcwY2xLGE)










Title: Re: A microcontroller programming language
Post by: DiTBho on December 17, 2022, 12:55:33 am
It sucks, but no way  :scared:
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 17, 2022, 01:21:08 am
Why stop at keywords, you could include operators too (unless you consider them keywords as well?) ;D
Title: Re: A microcontroller programming language
Post by: MIS42N on December 17, 2022, 08:38:32 am
Going back to the purpose of a programming language. I don't have a definition, but if asked I would say a language which translates intentions described by a human into instruction understood by a computer. Humans tend to ascribe particular meanings to particular words - in English 'if' 'when' 'do' 'then' 'next' all have meanings generally agreed on by people who speak the language. What is the problem reserving those words in a computer language to function somewhat like they do in the English language. I see no reason to allow a programmer to bend that.

What should be considered is what are common human intentions that need to be translated to computer speak. And since this discussion is about microcontrollers, one might add the condition "efficiently". There are plenty of languages that can create bloat, and for many application/processor combinations bloat can be tolerated. I was reading recently that micropython "is compact enough to fit and run within just 256k of code space and 16k of RAM". The BASIC in C64 et al was 8k of code for comparison, yet provided a reasonable subset of micropython capability. Much of the increase is not due to the language itself, but the inbuilt support libraries.

This is why I support the idea of adding computed goto, bit fields, packed decimal etc to the language. They are readily understood human constructs that need translating, their inclusion should enhance the chance that the compiler can generate more efficient code. After all, a computed goto in most languages could be written if 1 goto A1 else if 2 goto B1 else if 3 ...  - so there is no need for a specific computed goto construct, but it makes writing and reading easier.

But I think this discussion is likely to lead nowhere because millions of man hours and countless numbers of academics have pored over this problem for decades. What chance this discussion will have some breakthrough that wasn't considered before? I vote for 'fat' - but it would be nice to be wrong.

There are already languages designed for different purposes. I prefer assembler because I have plenty of time on my hands and I can churn out 20 or 30 lines an hour with a high chance it will work. Much of this is down to preparation - flow charts, modelling in another language (you maybe aghast to know my preferred is visual basic script), writing code that gets bashed rather than have dags that rarely if ever get called - e.g. lookup table rather than case statement. And documentation. Looking back at 10 lines of code I wrote weeks ago I can go WTF but then read the 3 or four lines of explanation and go oh, yeah - that's what it does.

Anyhow, just trying to derail this discussion about syntax and point back to why we have programming languages. I think that has been pushed down by trivialities.
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 17, 2022, 08:45:16 am
I can feel the power...

Code: [Select]
if if = then then

And I guess this is the first crucial step to enable your embedded friends write cycle-accurate code for PIC microcontrollers, yeah?
Title: Re: A microcontroller programming language
Post by: DC1MC on December 17, 2022, 09:17:25 am
I can feel the power...

Code: [Select]
if if = then then

And I guess this is the first crucial step to enable your embedded friends write cycle-accurate code for PIC microcontrollers, yeah?

Not only cycle accurate, but extremely easy to understand and maintain, not like the *x++ = **t-- C horror ?
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 10:11:01 am
Language v1.0 reserved words = if then else call end

Language v1.1 reserved words = if then else call end goto return

Language v2.0 reserved words = if then else call end goto return interrupt coroutine allocate

Language v3.4 reserved words = if then else call end goto return interrupt coroutine allocate lock trap retry

Does this make it clearer? We simply do not know what new features/keywords might get added in each future version of a language. Therefore if we add to the list of reserved words with each version we will eventually break older code somewhere, a v1.0 source file in part of a library, might have used "lock" as a procedure or function name, but refuses to compile under v3.4

There is nothing to stop us though from issuing style notices, or optionally telling a compiler (via some option) to act as if keywords are reserved and report errors, that's fine, that's a teams conscious choice though not an imposition by the language.

Being able to say "we might add zero or more new language keywords in each future version but guarantee that your existing source code will always compile, your code is guaranteed to compile under every future language version".

Names also must be explicitly declared, so unless one declares "if' and "then" and "else" there cannot be accidental confusion, the examples I've shown are test cases, proof that the grammar definition is sound, contrived examples used to test the grammar and parser, nothing more.

Reserved words are "training wheels", handcuffs to the language designers, anyone can write confusing looking code in any language, but that's their call, if they want to do it then that's their choice, C has plenty of reserved words but still let's us write insane looking code, keyword reservation is old fashioned thinking, time to move on.

Read this https://doc.rust-lang.org/reference/keywords.html



Title: Re: A microcontroller programming language
Post by: tggzzz on December 17, 2022, 10:28:45 am
Why not limit the keywords to being 6 pseudo-variables and nothing else? That allows you to have whatever control structures are suited to your requirements.

That's proven to be successful in embedded instruments manufactured by HP and Tektronix, e.g. protocol analysers and oscilloscopes and maybe more. I believe it to be significant that they were produced by hardware engineers (not software engineers) that only cared about how a language behaved and could achieve easily, and didn't care about the language itself.

The 6 words were/are: true, false, nil, self, super, thisContext.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 10:39:53 am
Why not limit the keywords to being 6 pseudo-variables and nothing else? That allows you to have whatever control structures are suited to your requirements.

That's proven to be successful in embedded instruments manufactured by HP and Tektronix, e.g. protocol analysers and oscilloscopes and maybe more. I believe it to be significant that they were produced by hardware engineers (not software engineers) that only cared about how a language behaved and could achieve easily, and didn't care about the language itself.

The 6 words were/are: true, false, nil, self, super, thisContext.

Smalltalk is listed here: https://stackoverflow.com/questions/4980766/reserved-keywords-count-by-programming-language

They left out PL/I and Fortran and a few others that are all 0.
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 17, 2022, 10:41:31 am
Not only cycle accurate, but extremely easy to understand and maintain, not like the *x++ = **t-- C horror ?

Think about it - in C, ***************************x is a valid expression, C bad.

"if if then if else then if if if if if then then if if if else then" is much better!
Title: Re: A microcontroller programming language
Post by: tggzzz on December 17, 2022, 10:51:08 am
Not only cycle accurate, but extremely easy to understand and maintain, not like the *x++ = **t-- C horror ?

Think about it - in C, ***************************x is a valid expression, C bad.

"if if then if else then if if if if if then then if if if else then" is much better!

No it isn't, since "all problems in computing can be solved by an extra level of indirection" :)

Similarly "all problems in signal processing can be solved by integrating for longer".
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 10:53:46 am
Not only cycle accurate, but extremely easy to understand and maintain, not like the *x++ = **t-- C horror ?

Think about it - in C, ***************************x is a valid expression, C bad.

"if if then if else then if if if if if then then if if if else then" is much better!

Except of course that the latter is syntactically wrong, illegal, gibberish; whereas the former is valid, legal C gibberish!
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 17, 2022, 11:23:35 am
Think about it - in C, ***************************x is a valid expression, C bad.

"if if then if else then if if if if if then then if if if else then" is much better!

Except of course that the latter is syntactically wrong, illegal, gibberish; whereas the former is valid, legal C gibberish!

What, your language uses whitespace as a syntactic separator? Why would you do that, isn't that limiting? What if the user wants to have spaces in the variable names, or what if you later find out you want keywords with spaces? I just wanted to have a variable named "if then if else then if if if if if then then if if if else" and you call it gibberish. Or wait, no, actually I want it with line feeds in variable names and all, how about:
Code: [Select]
   if if else then
      else if else then else
          then then =
      if then then


Clearly the language isn't powerful enough, yet! But I'm sure you'll get there, and the cycle-accurate PIC developers will thank you.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 11:30:52 am
Think about it - in C, ***************************x is a valid expression, C bad.

"if if then if else then if if if if if then then if if if else then" is much better!

Except of course that the latter is syntactically wrong, illegal, gibberish; whereas the former is valid, legal C gibberish!

What, your language uses whitespace as a syntactic separator? Why would you do that, isn't that limiting? What if the user wants to have spaces in the variable names, or what if you later find out you want keywords with spaces? I just wanted to have a variable named "if then if else then if if if if if then then if if if else" and you call it gibberish. Or wait, no, actually I want it with line feeds in variable names and all, how about:
Code: [Select]
   if if else then
      else if else then else
          then then =
      if then then


Clearly the language isn't powerful enough, yet! But I'm sure you'll get there.

Whereas you won't, if there's no grammar (formal syntax rules that can generate a parse tree) then it's not a language, its gibberish.

If you can define a grammar for your "language" I'll listen, until then...
Title: Re: A microcontroller programming language
Post by: DiTBho on December 17, 2022, 11:47:13 am
Incompetents do nothing but wasting people's time and resources.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 12:49:33 pm
Language v1.0 reserved words = if then else call end

Language v1.1 reserved words = if then else call end goto return

Language v2.0 reserved words = if then else call end goto return interrupt coroutine allocate

Language v3.4 reserved words = if then else call end goto return interrupt coroutine allocate lock trap retry

Does this make it clearer? We simply do not know what new features/keywords might get added in each future version of a language. Therefore if we add to the list of reserved words with each version we will eventually break older code somewhere, a v1.0 source file in part of a library, might have used "lock" as a procedure or function name, but refuses to compile under v3.4

There is nothing to stop us though from issuing style notices, or optionally telling a compiler (via some option) to act as if keywords are reserved and report errors, that's fine, that's a teams conscious choice though not an imposition by the language.

Being able to say "we might add zero or more new language keywords in each future version but guarantee that your existing source code will always compile, your code is guaranteed to compile under every future language version".

Names also must be explicitly declared, so unless one declares "if' and "then" and "else" there cannot be accidental confusion, the examples I've shown are test cases, proof that the grammar definition is sound, contrived examples used to test the grammar and parser, nothing more.

Reserved words are "training wheels", handcuffs to the language designers, anyone can write confusing looking code in any language, but that's their call, if they want to do it then that's their choice, C has plenty of reserved words but still let's us write insane looking code, keyword reservation is old fashioned thinking, time to move on.

Read this https://doc.rust-lang.org/reference/keywords.html

Yes, that does make it clearer.  I (also, if you do as well), also have much respect for Rusts, concerns and attempts/features, which attempts to take responsibility, for not only current source code, but older source code as well.  I.e. it hopes to allow current (or older) source, code to remain  compilable/compatible with new standards.  E.g. By allowing specific command version numbering syntax, to help with that process.

I think as already mentioned in this thread.  There is a fundamental problem, with new programming languages, in general.  They tend to have a fair amount of changes, in the early releases, as there is potentially much refining, polishing and fine honing/tuning of the language.  In the light of experience, and as more users, join the increasing family of people, using the new language.

I'm glad, you, early on are taking much thought and consideration, into this (apparently) important issue, with programming language design.

But there are other solutions.  Another would be to start off with a completed, reserved word list, which is set in stone.  Call it the 'hard' reserved word list.

For future releases of the language, any additions to the reserved list, can only be added to a 'soft' reserved word list.  The 'soft' version, unlike the 'hard' version, CAN BE over-ridden by the users of the compiler.  Unlike the original 'hard' reserved word list.

That way, you could get to eat your cake, and keep it.

Also, any obsolete 'hard' reserved words, could, instead of deleting them, be moved, at least for a long while, into  the 'soft' reserved word list.

'soft' reserved words, could still produce a compiler warning message.  With options to either treat it as an error, or ignore it completely.  As compiler flag options.  Then users could choose, how they want to treat the various reserved word lists.

Some languages, seem to use additional syntax, to allow many (rare) reserved words (or hidden variables/features, link/compiler information etc).  But because of the syntax, they will tend to never clash with programming source code.

E.g. using '_errorNumber' and '_errorDesription' to find out the compilers last run time, error codes/messages.  As they are less likely to be chosen names in the compiler, compared to e.g. 'error'.
E.g. Prefixing new words with an underscore or two. '_start' or '__start', or ending with a '!'. Etc.

Some languages (Python, I'm looking at you), change so much between, e.g. 2.0 and 3.0, that to all intents and purposes, it is a new and/or completely different (and relatively incompatible) language.  So, this does seem to be a REAL issue.

Even some/many frameworks/libraries and things, can change so much, between major revision numbers.  That there is massive incompatibilities, between them.  With various negative consequences, such as programmers who were familiar with (e.g. GTK2, compared to GTK3, and then GTK4), having later version(s), being significantly different, so they effectively have to re-learn it and rewrite/modify lots of code, in some cases.
It can annoy/alienate some/many people, so that they end up turning their backs on such systems.

You raise some good points, I'll think about what you suggest here. It would certainly be easy to make this stuff optional, or directive driven. For example we could envisage options like -reserved none, -reserved min, -reserved max or -reserved (if, then, else) that kind of thing.

Here 'min' means your 'soft' concept, reserves a core set of very common keywords but no more. The 'max' option on the other hand would reserve every keyword in the current version of the language, some people might value that, happy to see any clashes yet be able to still compile and change their code at a later time.

This is close to trivial to do, so long as the grammar itself imposes no restrictions, the compiler itself would do this. It must walk the parse tree anyway and can easily see if an identifier is also a keyword and then either do nothing or complain depending on the compile options.

I don't think some people here grasp the fact that if the grammar itself has reserved words then you're locked in fundamentally, reserving words outside of the grammar during one of the compilation phases is much much better.

So I do very much like your idea.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 01:05:23 pm
There are potentially a lot of keywords too, so reserving all is a big burden cognitively on the programmer. This is just some of the keywords I'm expecting to have:

if then else while end proc procedure func function, dcl declare, bin binary dec decimal fixed float complex cplx def define call return coroutine cofunction yield to by when static auto automatic based defined ptr pointer allocate bit string byte free goto go to

You get the idea, the fact is this set can grow over time (new language features, attributes, data types etc) without any worry or concern we might break some user's code base, sure some users might want to reserve some or all (current) keywords, but that's their call and they should be given that option.

Imagine you were calculating the yield of some process:

Code: [Select]

yield = ((total - defects) / total) * 100;


Then one day you upgrade to v3.4 of newlang only to find your code no longer compiles because newlang now supports iterators and 'yield' can no longer be used that way! Right away we must change code, run unit test, create commits, review code and so on, a total waste of people's valuable time all imposed by a language because the designers didn't think about backward compatibility.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 03:30:35 pm
Then one day you upgrade to v3.4 of newlang only to find your code no longer compiles because newlang now supports iterators and 'yield' can no longer be used that way! Right away we must change code, run unit test, create commits, review code and so on, a total waste of people's valuable time all imposed by a language because the designers didn't think about backward compatibility.


It is not just that.  When such changes occur, significantly in the future.  The original programmers, may be unavailable (busy on other projects), or have left the company.  Even if available, it might be many years ago, when they wrote that code, so they have totally forgotten about it.
So, for them (or others), to now go back, and rework the source code.  Can be a rather time-consuming, problematic (further new bugs might be introduced, because they no longer understand the code), and an expensive loss of (valuable/limited) resources, that could be better spent, doing other useful things.

It might even be someone, trying to recompile the software, and sort out any dependencies, just so they can use it, some time in the future.  So if the (then) current V3.4, no longer compiles, without many fatal errors, using the existing source code base (which could be open source or commercial).  That will put a real damper, on the activities, within the new language.

Some (very successful and popular) programming languages, have such huge existing software bases, in source format.  That making future changes to the language, without breaking any existing functionality.  Can be a real-minefield.

Taking CPU instruction sets, as an analogy.  On the one hand, the X86 / X86-64 instruction sets, have amazing compatibility, even going back around 40 years, to the original 8086/88 CPUs.  But some people think, that it also causes big baggage (like) problems.  Whereas alternative architectures, such as Arm, tend to be rewritten from scratch, only going back one or two instruction sets, as regards compatibility.  But that potentially allows the instruction set to move on from the old days, and soak up, any modern CPU performance improvements, with minimal handicapping, from old, somewhat obsolete instruction sets, of the distant past.

E.g. How many people, really run native 8088 programs, on their 192 core (dual socket 2 x 96 core), AMD Epic processors?
I have heard some of the backward compatibility, may be no longer available, but I'm not sure.  Removing it has been discussed, but I'm not sure if it happened or not.
Windows, seems to happily drop support, for even rather modern/recent processors.  But that could be, because they get paid license fees, mainly when new PCs are purchased, so it is in their interests, to obsolete CPUs, even if they are rather recent ones.

In other words.  It is annoying when a programming language, introduces, show breaking changes, between say versions 2.0 and 3.0 .  But, that might (or might not), be worth it, from a longer time scale point of view.

E.g. Despite Python, sometimes dramatically changing, between major release number versions.  It does seem to still be very popular.  So I wouldn't like to say, one method is best.  It is a tricky situation to handle.
As they say, you can't make a omelette without breaking eggs.  Maybe such dramatic changes, is ok, if done responsibly, and only very rarely.  Such as once every 7 or 8 years.

If you DON'T allow show stopping changes, ever.  Eventually the thing (a programming language in this case), can eventually become stale and fixed.  Unable to move forward, with any/much momentum.  Ultimately, risking the language becoming obsolete (i.e. people stop using it), as time goes on.

If you are still primarily aiming this to be an MCU language.  There is a tendency for such projects, to have relatively short development time-scales (but, they COULD be very long, it depends).  With whatever language version, they started out with, on that particular MCU, remaining, throughout the project life-time.
Because if you have MCU (embedded) software, which works well and is considered bug free.  Made with version 1.24 of the compiler, nine years ago.
Experience tends to find, that if you update the compiler to the latest version 9.79, on a much later version of windows.  There is a tendency, for various bugs (that didn't exist previously), to suddenly appear, mismatches within the new (updated) libraries and even entire functionality, that is no longer present in the latest libraries that come from the manufacturer.
The latest compiler, may not even work, with your MCU, from many years ago.

If it ain't broke, don't mess with it.

With FPGAs, and older specific FPGAs, typically.  You can't even use, older FPGAs, on the latest FPGA software packages, as support was dropped for those older FPGAs, many years/versions ago.

You started off this thread with mention of PIC MCUs.  Which tend to lose support for the older PIC models, on the latest programmer-devices/compilers and libraries.  So, total backward compatibility, may not be necessary or even a good idea.  I suppose care/caution is needed.

Also remember the K.I.S.S. (Keep It Simple Silly/stupid), principal.  Making the compiler too overly complicated, with way too much functionality.  May mean that, it can never be completed, given its available resources (such as programming team size/time, for creating the compiler).  Sometimes, you have to harshly remove extra functionality, to make things practicable.


I did start the thread with an emphasis on MCU applications, that's true. Perhaps I should explain/clarify what that means. To me, it means a compiled language that includes features that although having general world use, are particularly helpful to the hardware programmer. In other words a language that can be attractive and linguistically contends with C and comparable languages used today.

For example explicit support for a bit data type, operators for arith-shift, logic-shift, several ways of overlaying types in memory, offset as well as traditional pointers and so on.

A little bit about me too, might also help understand my position here, I apologize for the length and the apparent self indulgence. I was an electronics hobbyist as a kid, when I lived in Liverpool UK. I was about 14 or so when a mate introduced me to transistors, radio and stuff. I quickly became engrossed and he and I did several projects together in the mid/late 1970s.

We managed to get stuff published in Everyday Electronics too, a bit of cash but a fun experience for two teenagers. I soon became absorbed by microprocessors after they became huge in magazines like Practical Wireless and ETI, I bought a basic 6502 kit from Acorn and interfaced that to motors and all sorts.

By the age of like 18/19 I got into a specialist electronics college (my mate got a place at Riversdale college in Liverpool and studied marine radio and radar and ended up doing defense work overseas for a few years) and for the next two years studied (full time) electronics, DC/AC, passive circuits, transistors, amplification, RF, telephone systems, power/motors etc, microprocessors, digital systems, lots of hands on work with scopes, soldering and stuff, a pretty dense two year education, approx at least the first year of an EE degree IMHO.

After I finished I did some more writing for about a year, just living an ordinary life in Liverpool and still reading everything I could find on electronics and computing, this is around 1980/1981.

I started to look for a job - hugely wanted an electronics job, I had very good marks from college, several published articles in my portfolio and so on, but could not find work, Liverpool was depressed, high unemployment, factories closing and a few jobs I did interview for outside of Liverpool never materialized, I think not having a proper degree was a handicap too, despite my definite competence and skills.

By fluke a friend mentioned a new 14 week government run course in Liverpool, taught by a private firm, subsidized by the government to deal with high unemployment. Two hundred applied for twenty places, I was one of those that got through the lengthy test process.

I was asked to choose between training in IBM Assembler, COBOL and PL/I. A friend said "PL/I's a pretty serious language, great for writing systemy stuff" so I said OK what the heck and chose that.

Toward the end the firm made efforts to get us interviews, I got one in Enfield, London and they offered me a job, so I left home and started working for the local government as a trainee PL/I programmer, it was nice, decent work, lots of older, experienced clever people and several lads about my age all interested in microprocessors, BASIC, assembler, electronics, Apple etc.

I learned a lot about large scale projects, working in a team, paying attention to detail, communicating technically with others, I was liked and I liked the place.

I ended up working as a software guy thereafter, my 'dream' of a serious electronics career faded and I became a "developer" and am to this day.

Now, as a trained, educated, intelligent electronics person I was interested to see PL/I's features for bits and storage overlays and pointers and so on, I was very at home with technically oriented software, people gave me "technical" work because I had insights that only an electronics engineer would have,

I took it for granted that many other languages also supported stuff like bits, pointers, offsets etc but slowly realized they did not, as I learned C and other languages I recall thinking "Man, this is crazy, that older mainframe language is way beyond these other languages and it would be much better to write stuff in PL/I than C" and so on.

In a word, most other languages looked feeble, rushed, poorly thought out, clumsy, to me anyway, and recall I was not and am not a lightweight in electronics.

So that's my background and so when I discuss programming languages and MCU programming, I'm not naive as some here might want to think, I am inexperienced specifically with these devices and don't pretend to be an expert but I do know how to develop software, I do understand electronics, I do understand compilers, machine language, assembler, and so on.

So this "new language" if I do take it to the next level, is a very serious subject to me, and I see a place for a language that specifically supports features that will aid in MCU development work, I'm 100% definite about that. It's also very interesting, I've built a large compiler already in the past (and wrote it in C) so I am well aware of what's involved.

The PIC stuff was just comments I'd picked up on other forums, PIC developers griping about how hard it is to use C, it seems PIC is an outlier anyway so nothing to be gained by dwelling on that.







Title: Re: A microcontroller programming language
Post by: DiTBho on December 17, 2022, 03:56:55 pm
whatever

Imagine you were calculating the yield of some process:

Code: [Select]

yield = ((total - defects) / total) * 100;


Then one day you upgrade to v3.4 of newlang only to find your code no longer compiles because newlang now supports iterators and 'yield' can no longer be used that way! Right away we must change code, run unit test, create commits, review code and so on, a total waste of people's valuable time all imposed by a language because the designers didn't think about backward compatibility.

less than two minutes to fix this  :-DD

Title: Re: A microcontroller programming language
Post by: Siwastaja on December 17, 2022, 04:33:00 pm
Variable renaming is simple work, you don't need the original developer to do it.

If you are changing to different language version, because you want to use new features (so you will be using the new features, and modifying code, or writing new code), a simple variable renaming is going to be smallest of your problems. You can't avoid unit-testing and testing the whole thing with all the changes.

And if you are not going to make any changes, why change the language version then?

(See, this is the difference between real and made-up problems. Real problems can be difficult, but made-up problems are ridiculous when you look at them closely.)
Title: Re: A microcontroller programming language
Post by: DiTBho on December 17, 2022, 04:48:55 pm
(See, this is the difference between real and made-up problems. Real problems can be difficult, but made-up problems are ridiculous when you look at them closely.)

(indeed)
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 17, 2022, 04:56:04 pm
its called delusion... and i dont get this "bit field support" delusion in newer languague. C already support it, its the job of compiler to make best use of any available machine specific codes for it.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 17, 2022, 04:58:02 pm
Variable renaming is simple work, you don't need the original developer to do it.
 

Macros can make that non-trivial. In a decent language the source code contains enough information that  an IDE can refactoring - either the declared name or every place the variable name is used.

Quote
If you are changing to different language version, because you want to use new features (so you will be using the new features, and modifying code, or writing new code), a simple variable renaming is going to be smallest of your problems. You can't avoid unit-testing and testing the whole thing with all the changes.

And if you are not going to make any changes, why change the language version then?

(See, this is the difference between real and made-up problems. Real problems can be difficult, but made-up problems are ridiculous when you look at them closely.)

Sometimes you have to change because a component within your system has changed and must be updated.

Sometimes you have to change because a component outside your system has changecd, either the API signature of the data passed.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 05:15:40 pm
whatever

Imagine you were calculating the yield of some process:

Code: [Select]

yield = ((total - defects) / total) * 100;


Then one day you upgrade to v3.4 of newlang only to find your code no longer compiles because newlang now supports iterators and 'yield' can no longer be used that way! Right away we must change code, run unit test, create commits, review code and so on, a total waste of people's valuable time all imposed by a language because the designers didn't think about backward compatibility.

less than two minutes to fix this  :-DD

Why break it in the first place.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 05:32:46 pm
Variable renaming is simple work, you don't need the original developer to do it.

If you are changing to different language version, because you want to use new features (so you will be using the new features, and modifying code, or writing new code), a simple variable renaming is going to be smallest of your problems. You can't avoid unit-testing and testing the whole thing with all the changes.

And if you are not going to make any changes, why change the language version then?

(See, this is the difference between real and made-up problems. Real problems can be difficult, but made-up problems are ridiculous when you look at them closely.)

Imagine we have twenty source files, all comprising some stable deliverable, it works, its tested.

We want to leverage new async support (say) and that means we'll be changing the implementation of source file number 6, so we do that, we edit it and make the change.

Then we try to build the app again and find it fails to build, what was formerly valid code is now invalid, what was formerly free of syntax errors, now contains syntax errors. Worse still the offending code is in an include file used by twenty other unrelated projects, now we can't build any of those projects without changing that shared file, unless we remember to only use the new version of the compiler on this one file, yes, that'll work...

So we edit that include file and change the function in there (that was named 'async') to some new name like 'basync', all good, we run our rebuild.

But oh dear, there are umpteen places where that function 'async' was being called and include files too, so now we must edit fifteen files in unrelated projects that are referring to a function that we just renamed, but we first have to identify all those unrelated projects  - oh dear.

Even worse, recall all that documentation you wrote and shared with the overseas office? the documentation that explained the design of the function named 'async', you know, that core function that everything hinges on, well you'll have to edit and update all of the documentation and send emails to everyone that might be impacted because you're on vacation next week yet Jane in Spain is due to take the code for a demo, the code that she'll be referring to in her PowerPoint slide show to the important German customer, where the PowerPoint slides discuss the important function named 'async'.

Its at this point that most seasoned developers will say "WTF, why did they have to add that new keyword and cause all this hassle, what numpties are these that develop these languages".

See? this is the difference between real software development and pretend software development, where time, costs, impact don't matter - which is ridiculous of course isn't it...




Title: Re: A microcontroller programming language
Post by: DC1MC on December 17, 2022, 05:39:39 pm
Well, I'm a simple down to earth guy, not an architecture astronaut, so if some one has time, skills and interest in improving a language, why not not get a pre-existing C compiler, a simple version like TinyCC, or LCC, or even SDCC and implement some of the proposed feature in the first posts instead of going into demented syntax and paradigms from long lost languages suitable for long lost platforms and that they were proved hard to use and mostly useless at the time when they were hot and teached in schools and universities.
You know these paradigms that promise that if you throw away all your experience and way of thinking, followed by all previous source code and libraries, you'll get some new and vastly improved way of doing things, one has only to re-learn everything and sky is the limit. This (IMHO) failed before and will fail now as well.
Instead let's implement the low hanging fruits properly in C, I'l put here the low-hanging fruits that I would like to have in a C compiler (extension) and that doesn't need a new grammar or crazy syntax:

- binary representation of numbers: make 0y11_101_1_00 possible, with _ as group separator to make life easier, add some common sense rules of filling values depending of the left size and throwing warning or errors if there are more bits in the source as in the destination,

- introduce a:
#pragma cycle_critical start <max cycle number>
code..
#pragma cycle_critical stop
to enable sections of code that are critical, if the back-end knows how to generate such a code it will do best effort to fit the code in the number of given cycles, or spit out warnings/errors like: "WARNING: Implementation possible in XXX cpu cycles, offered YYY...", if not it will be just ignored (maybe with warnings) to keep compatibility.

- computed gotos, here I don't know how to offer a proper syntax, maybe use @labels everywhere and put them in an enum/pointers array or something that can be used for calculations ?

If somebody will be able to implement this in a sane way in one of the existing C/C++ compilers suitable for MCUs (see above) it will definitely get my full respect, maybe even some shekels if it works nicely and the result will even have a chance to be used by more than two people  ^-^. Otherwise I feel the will be yet another implementation of brainfuck.

Just my two humble € cents, now three fiddy because of inflation.

 Cheers,
 DC1MC


Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 05:43:01 pm
its called delusion... and i dont get this "bit field support" delusion in newer languague. C already support it, its the job of compiler to make best use of any available machine specific codes for it.

Not this again. Try reading carefully what I write. I said that C "has no bit data type" I did not say C has no support for bit fields. Yes you can write almost anything in C, I know that I wrote a large compiler in C, remember? You can write anything in assembler too, is that an argument for not using C?

Lets paraphrase what you just wrote:

Quote
Assembly language already support it, its the job of assembler to make best use of any available machine specific codes for it.

If your satisfied and content using C then I'm happy for you, this thread is about the issues and features that emerge when discussing what a new better language might look like, might do.


Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 05:47:54 pm
Well, I'm a simple down to earth guy, not an architecture astronaut, so if some one has time, skills and interest in improving a language, why not not get a pre-existing C compiler, a simple version like TinyCC, or LCC, or even SDCC and implement some of the proposed feature in the first posts instead of going into demented syntax and paradigms from long lost languages suitable for long lost platforms and that they were proved hard to use and mostly useless at the time when they were hot and teached in schools and universities.
You know these paradigms that promise that if you throw away all your experience and way of thinking, followed by all previous source code and libraries, you'll get some new and vastly improved way of doing things, one has only to re-learn everything and sky is the limit. This (IMHO) failed before and will fail now as well.
Instead let's implement the low hanging fruits properly in C, I'l put here the low-hanging fruits that I would like to have in a C compiler (extension) and that doesn't need a new grammar or crazy syntax:

- binary representation of numbers: make 0y11_101_1_00 possible, with _ as group separator to make life easier, add some common sense rules of filling values depending of the left size and throwing warning or errors if there are more bits in the source as in the destination,

- introduce a:
#pragma cycle_critical start <max cycle number>
code..
#pragma cycle_critical stop
to enable sections of code that are critical, if the back-end knows how to generate such a code it will do best effort to fit the code in the number of given cycles, or spit out warnings/errors like: "WARNING: Implementation possible in XXX cpu cycles, offered YYY...", if not it will be just ignored (maybe with warnings) to keep compatibility.

- computed gotos, here I don't know how to offer a proper syntax, maybe use @labels everywhere and put them in an enum/pointers array or something that can be used for calculations ?

If somebody will be able to implement this in a sane way in one of the existing C/C++ compilers suitable for MCUs (see above) it will definitely get my full respect, maybe even some shekels if it works nicely and the result will even have a chance to be used by more than two people  ^-^. Otherwise I feel the will be yet another implementation of brainfuck.

Just my two humble € cents, now three fiddy because of inflation.

 Cheers,
 DC1MC

Sure that's an option, just grab C and its broken inflexible grammar and add to the mess, that's what they did to get C++, see how well that worked out? today considered the most difficult language to parse and extend.

Take a look at this

Quote
C++03's parser defines “>>” as the right shift operator or stream extraction operator in all cases. However, with nested template declarations, there is a tendency for the programmer to neglect to place a space between the two right angle brackets, thus causing a compiler syntax error.

Get that? until 2011 one had to use a space in stuff like this:

Code: [Select]
List<Frame<int> > items;

Omit that space and OH BOY, the parser sees a right shift operator and all hell breaks lose...




As for "demented syntax" you'll need to formally define that for me, might you actually mean unfamiliar perhaps?

Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 17, 2022, 05:50:43 pm
Quote
Assembly language already support it, its the job of assembler to make best use of any available machine specific codes for it.
no! assembly is 1 to 1 map to machine code, no need to translate in any fancier way. assembler is just that, 1 to 1 mapping.. duh! how you can make such conclusion?
Title: Re: A microcontroller programming language
Post by: DiTBho on December 17, 2022, 05:54:05 pm
if some one has time, skills and interest in improving a language, why not not get a pre-existing C compiler

That's precisely what I did with my-c :D
So, it can be done!
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 05:57:09 pm
Quote
Assembly language already support it, its the job of assembler to make best use of any available machine specific codes for it.
no! assembly is 1 to 1 map to machine code, no need to translate in any fancier way. assembler is just that, 1 to 1 mapping.. duh! how you can make such conclusion?

It's a language too, assembler can do anything (and more) that C can do.
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 17, 2022, 05:59:49 pm
if some one has time, skills and interest in improving a language, why not not get a pre-existing C compiler
That's precisely what I did with my-c :D
So, it can be done!
can you publish for discussion? its much better than discussing uthopia. keep re-stating something secret will not do any good, similar to uthopia story..
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 17, 2022, 06:03:18 pm
Quote
Assembly language already support it, its the job of assembler to make best use of any available machine specific codes for it.
no! assembly is 1 to 1 map to machine code, no need to translate in any fancier way. assembler is just that, 1 to 1 mapping.. duh! how you can make such conclusion?
It's a language too, assembler can do anything (and more) that C can do.
and havent you got the idea yet? its a 1 to 1 mapping from brainf*ck grammar to at least something human can cope... so instead of typing 0x0001F 10E, we can type MOV REG1 for example.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 06:04:41 pm
Consider:

Quote
Unlike Java, which has a special operator >>> for logical shifting apart from the usual >> and <<, C and C++ have only arithmetic shifting with some areas left undefined and implementation-defined.

Most CPU architectures include instructions for arithmetic shift, logical shift and rotates, yet C that "designed for hardware" language doesn't support most of these variants, don't you get it? it was never designed for the applications you keep claiming it was designed for, it wasn't even designed if I can be blunt.


Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 06:06:00 pm
Quote
Assembly language already support it, its the job of assembler to make best use of any available machine specific codes for it.
no! assembly is 1 to 1 map to machine code, no need to translate in any fancier way. assembler is just that, 1 to 1 mapping.. duh! how you can make such conclusion?
It's a language too, assembler can do anything (and more) that C can do.
and havent you got the idea yet? its a 1 to 1 mapping from brainf*ck grammar to at least something human can cope... so instead of typing 0x0001F 10E, we can type MOV REG1 for example.

Yes I know that, but it is a language isn't it? are you claiming it's not a language? has no syntax rules? are you saying that?

Title: Re: A microcontroller programming language
Post by: DC1MC on December 17, 2022, 06:06:47 pm
Tja, I've tried to suggest something way easy to implement IMHO (that is: doable) than a whole new language, but I guess that slowly, slowly this thread will either goto (sic) irrelevance or deeper into d1ck measuring contest with more and more impractical stuff.
Out of everything I think DiTBho got the best outcome, at least they (see, political correct addressing mode  >:D) got something that is usable and makes their (again :-DD) life easier.

The whole syntax boogaloo of C++ is because they've tried to cram as way much and outside of C paradigm stuff into it, but still it become one of the most used and popular languages, just because one doesn't need to throw away everything what they know and are familiar with, as well as the existing code base just to get some extra stuff.

But by all means, let me know when your improved language generates code, I will give it a fair try.

 Cheers,
 DC1MC
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 17, 2022, 06:17:46 pm
Quote
Assembly language already support it, its the job of assembler to make best use of any available machine specific codes for it.
no! assembly is 1 to 1 map to machine code, no need to translate in any fancier way. assembler is just that, 1 to 1 mapping.. duh! how you can make such conclusion?
It's a language too, assembler can do anything (and more) that C can do.
and havent you got the idea yet? its a 1 to 1 mapping from brainf*ck grammar to at least something human can cope... so instead of typing 0x0001F 10E, we can type MOV REG1 for example.
Yes I know that, but it is a language isn't it? are you claiming it's not a language? has no syntax rules? are you saying that?
there is difference between human language vs machine language... the purpose of the former is to fill every gap way down to the later. the way i see whenever anyone want to create a new language, its the human (friendly) language, a higher level one... but if what you are after is brainf*ck language, then by all means go for it, i've provided a link for you earlier to win the contest... just make it clear to us..
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 06:37:38 pm
Quote
Assembly language already support it, its the job of assembler to make best use of any available machine specific codes for it.
no! assembly is 1 to 1 map to machine code, no need to translate in any fancier way. assembler is just that, 1 to 1 mapping.. duh! how you can make such conclusion?
It's a language too, assembler can do anything (and more) that C can do.
and havent you got the idea yet? its a 1 to 1 mapping from brainf*ck grammar to at least something human can cope... so instead of typing 0x0001F 10E, we can type MOV REG1 for example.
Yes I know that, but it is a language isn't it? are you claiming it's not a language? has no syntax rules? are you saying that?
there is difference between human language vs machine language... the purpose of the former is to fill every gap way down to the later. the way i see whenever anyone want to create a new language, its the human (friendly) language, a higher level one... but if what you are after is brainf*ck language, then by all means go for it, i've provided a link for you earlier to win the contest... just make it clear to us..

So assembler is a language then, good we agree on that. Now can the C language do anything that assembly language can't? You want to talk about languages, lets talk...
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 17, 2022, 06:58:32 pm
...snip..
So assembler is a language then, good we agree on that. Now can the C language do anything that assembly language can't? You want to talk about languages, lets talk...
in the end... your brainf*ck language will get eventually be translated into machine codes anyway... that turing complete machine?... like all other programming languages in existance. you want bit type and string support? go for it! it will get introduced into bloatness of machine codes anyway... like most of the rest (Basic, Pascal etc)... and sorry, i'm not further discussing since its clear what you re after now (brainf*ck language however powerful it is) and my first reply has become more and more apparent (getting worse actually). it has no practical use for me... if you've made it clear since your first post, i would've not make the first post, not even a glimpse. but the ship had sail, this thread will be a curse for me appearing whenever i hit "Show new replies to your posts". tldr.... only one thing i ask from you.... please make it short! pleaseeee! ;D (ps: i have another 1 or 2 never ending cursed threads going on that keep posting no practical knowledge to me)... cheers.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 06:58:56 pm
...snip..
So assembler is a language then, good we agree on that. Now can the C language do anything that assembly language can't? You want to talk about languages, lets talk...
in the end... your brainf*ck language will get eventually be translated into machine codes anyway... that turing complete machine?... like all other programming languages in existance. you want bit type and string support? go for it! it will get introduced into bloatness of machine codes anyway... like most of the rest (Basic, Pascal etc)... and sorry, i'm not further discussing since its clear what you re after now (brainf*ck language however powerful it is) and my first reply has become more and more apparent (getting worse actually). it has no practical use for me... if you've made it clear since your first post, i would've not make the first post, not even a glimpse. but the ship had sail, this thread will be a curse for me appearing whenever i hit "Show new replies to your posts". tldr.... only one thing i ask from you.... please make it short! pleaseeee! ;D

Just let me know when you'd like to continue actually discussing programming languages, I'll be here.

Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 17, 2022, 07:13:48 pm
i'll be here! ;D i thought you are the one asking for discussion? instead of posting proposal thats becoming more and more resembling Brainfuck (https://en.wikipedia.org/wiki/Brainfuck) or just talk like...

https://www.youtube.com/watch?v=yL_-1d9OSdk (https://www.youtube.com/watch?v=yL_-1d9OSdk)

at least you can relieve big stress out of people... ;D
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 07:33:16 pm
i'll be here! ;D i thought you are the one asking for discussion? instead of posting proposal thats becoming more and more resembling Brainfuck (https://en.wikipedia.org/wiki/Brainfuck) or just talk like...

https://www.youtube.com/watch?v=yL_-1d9OSdk (https://www.youtube.com/watch?v=yL_-1d9OSdk)

at least you can relieve big stress out of people... ;D

I asked for suggestions about things, features people would like to see in a new language if we were designing a new language and we wanted that language to help with development on MCUs.

A few people in the thread have understood that and politely contributed, but some others have not and prefer to object to the sheer effrontery in even hinting that anything better than C is possible.
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 17, 2022, 07:54:02 pm
..but some others have not and prefer to object to the sheer effrontery in even hinting that anything better than C is possible.
maybe for your own good? certainly anything is not impossible, we just open to that, but we dont know how.. since you think you can do it, just prove it in short possible time, not suggesting far more worse example from your wonderland delusion.

remember. when you are done, you are going to do a great length documenting it to make people understand. and thats where peer reviews/discussion will actually take place... and know how many takers and nayers you are going to get.. that number will means something... at this stage.. we just a bunch of blocking stone to you (we tried not to be though), some are just drawing their honest opinions, even if you dont like it.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 17, 2022, 08:27:44 pm
..but some others have not and prefer to object to the sheer effrontery in even hinting that anything better than C is possible.
maybe for your own good? certainly anything is not impossible, we just open to that, but we dont know how.. since you think you can do it, just prove it in short possible time, not suggesting far more worse example from your wonderland delusion.

remember. when you are done, you are going to do a great length documenting it to make people understand. and thats where peer reviews/discussion will actually take place... and know how many takers and nayers you are going to get.. that number will means something... at this stage.. we just a bunch of blocking stone to you (we tried not to be though), some are just drawing their honest opinions, even if you dont like it.

"Do" what? I'm seeking to discuss the subject not bicker, I'll progress this at my own speed, I've got nothing to prove to you or anyone else, I'm not seeking to discuss me, my abilities, how fast I work, or any of that stuff, you want to talk about me all the time, I'm done interacting with you, have a nice day.
Title: Re: A microcontroller programming language
Post by: MIS42N on December 17, 2022, 09:34:02 pm
Or tell the compiler #operator async disable. Since the legacy code didn't use async as a reserved word, this works.

My background was programming in multiple languages cobol fortran c basic various job control languages and a pile of assemblers ibm 360, icl 1900, pdp 8, pdp 11, 8086, 6502, z80, pic 8 bit, avr 8 bit and a few more that I've forgotten. Language is pretty much irrelevant, like human languages. Can express a concept in any of them. For fun I've written programs using subleq, just to satisfy this thought. It doesn't take a lot to come up with a set of macros that look like instructions in a higher language, then use them.

So I am much more interested in what the proposed language can do rather than how it is expressed. Good documentation offsets most of the obfuscation present in any language, and more importantly, what the programmer was trying to achieve (which often is not obvious from the code regardless of how clear the language). Let's move on from this syntax discussion. Let the OP have a language with no reserved words, we can live with that anomaly even if we don't want it. Finish the entree, move on to the main meal.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 17, 2022, 10:21:12 pm
My background was programming in multiple languages cobol fortran c basic various job control languages and a pile of assemblers ibm 360, icl 1900, pdp 8, pdp 11, 8086, 6502, z80, pic 8 bit, avr 8 bit and a few more that I've forgotten. Language is pretty much irrelevant, like human languages. Can express a concept in any of them. For fun I've written programs using subleq, just to satisfy this thought. It doesn't take a lot to come up with a set of macros that look like instructions in a higher language, then use them.

Ditto, and yes.

Plus the first language I used professionally was such a macro language cross-compiled on a PDP-11 for a 6800 target.

Quote
So I am much more interested in what the proposed language can do rather than how it is expressed.
...
Let's move on from this syntax discussion.

Exactly, but the OP has repeatedly ignored those points in favour of discussing grammars, parse trees and similar compiler constructs.

That's fine, but won't get any further than people wanting to build their own processor.
Title: Re: A microcontroller programming language
Post by: MIS42N on December 17, 2022, 11:57:00 pm
Interpreted languages and LLVM are OK to a point. But when dealing with time constrained hardware problems with unpredictable inputs it just doesn't work. A few years ago I wrote a top octave generator using an Arduino nano (16MHz clock, ATMega328 CPU). It simultaneously produced 12 square waves on 12 pins with frequencies from 4435Hz to 8368Hz with transitions to the nearest 500ns. It would be interesting to see what processor could achieve that with an interpreter. Or even with a compiler. The object code was huge but in the main generated by assembler macros so the source was not so big.

I use interpreted language to test modules. Quick development, plenty of ways to generate input and test output. But then hand translate to the target language for compilation. I have done the reverse, hand translate from IBM assembler to C because nobody else could support an IBM assembler program. Another case of it doen't matter what language, can express an intent in any (as long as there's not timing constraints).
Title: Re: A microcontroller programming language
Post by: DiTBho on December 18, 2022, 12:33:30 am
A few years ago I wrote a top octave generator using an Arduino nano (16MHz clock, ATMega328 CPU). It simultaneously produced 12 square waves on 12 pins with frequencies from 4435Hz to 8368Hz with transitions to the nearest 500ns. It would be interesting to see what processor could achieve that with an interpreter. Or even with a compiler. The object code was huge but in the main generated by assembler macros

So, you wrote it in assembly   :o

Some mpus have hw helpers.
Special programmable circuits that only need glue code.

my-c does not optimize the generated assembly code
It's a problem if you need performance.
But it's not its purpose.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 18, 2022, 02:52:16 am
Or tell the compiler #operator async disable. Since the legacy code didn't use async as a reserved word, this works.

My background was programming in multiple languages cobol fortran c basic various job control languages and a pile of assemblers ibm 360, icl 1900, pdp 8, pdp 11, 8086, 6502, z80, pic 8 bit, avr 8 bit and a few more that I've forgotten. Language is pretty much irrelevant, like human languages. Can express a concept in any of them. For fun I've written programs using subleq, just to satisfy this thought. It doesn't take a lot to come up with a set of macros that look like instructions in a higher language, then use them.

So I am much more interested in what the proposed language can do rather than how it is expressed. Good documentation offsets most of the obfuscation present in any language, and more importantly, what the programmer was trying to achieve (which often is not obvious from the code regardless of how clear the language). Let's move on from this syntax discussion. Let the OP have a language with no reserved words, we can live with that anomaly even if we don't want it. Finish the entree, move on to the main meal.

I'm not proposing a language, go on, check all the posts for yourself. You can't talk about language as distinct from "how it is expressed", that IS LANGUAGE. We can't even discuss languages without using a language, anyway, I'm tired, good night.
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 18, 2022, 06:25:54 am
Most CPU architectures include instructions for arithmetic shift, logical shift and rotates, yet C that "designed for hardware" language doesn't support most of these variants, don't you get it? it was never designed for the applications you keep claiming it was designed for, it wasn't even designed if I can be blunt.
As already touched upon, by some in this thread.  You may have to be careful, if you support, too low a programming level, in a programming language.  It could end up being counter-productive.
Mr Ritchie probably has figured it out decades ago, so by realizing this... there is escape plan to wash C's hands off... assembly-program-in-c (https://www.javatpoint.com/assembly-program-in-c) so its impossible in C, only whenever the assembly/machine code make it impossible (unavailable) including THAT arithmetic/logical shift... this is also probably why the C got the bashing of having so many different standards across platforms... want to invent a unified language for all at the very low level? good luck!
Title: Re: A microcontroller programming language
Post by: DiTBho on December 18, 2022, 12:06:19 pm
And don't forget tr-mem!
It's something rare at the moment ( POWER10 had it, how many users do use GNU/Linux POWER10? ), but I think it will be the future in multi-core

Special atomic memory model meeded
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 18, 2022, 12:24:48 pm
Most CPU architectures include instructions for arithmetic shift, logical shift and rotates, yet C that "designed for hardware" language doesn't support most of these variants, don't you get it? it was never designed for the applications you keep claiming it was designed for, it wasn't even designed if I can be blunt.

As already touched upon, by some in this thread.  You may have to be careful, if you support, too low a programming level, in a programming language.  It could end up being counter-productive.

Let's say, a very low level instruction set, is included as part of the high level language, but it is TOO low a level.

So it might include (just an example, other instructions could have similar, unintended consequences, this instruction has already been mentioned in this thread).
A subtract (assembly like) instruction as part of the high level language, complete with a functionally defined borrow flag, over-flow detection and maybe some other bits and pieces, such as a zero (all zeros) flag.

One or more actual CPUs the new programming language targets, may have this very instruction, so that is fine.

But any other supported architectures (CPUs), may have significantly different subtract instructions.  They might use a carry, rather than a borrow flag, and its definitions could vary.  There might not be an overflow flag, or the way it works could vary.  The zero flag, may be set under different conditions, e.g. index registers may or may not, affect the zero flag, etc.

So, in order to implement the new programming language, and have consistent results, between differing architectures.  What would have been a single subtract (machine code) instruction, in one of the supported architectures.  Has to be replaced, by lots of instructions, just to emulate the functionality, of a different architecture.

So, for the original/primary supported archetecture, the following program, might be replaced with a single (fast) instruction.

a = a - 2;
Becomes, after compiling:
Subtract-with-borrow 2

But on other archetectures, with completely differing subtract machine code instructions and flag setups.  It might look like this:
a = a - 2;
becomes, after compiling:
clear-carry-flag
Subtract-with-carry 2;
Instructions, including branches to invert the carry flag
Instructions to calculate the overflow status
Other instructions to emulate the original subtract instruction, shown above, in the first example
Etc.

So instead of being a fast, single instruction (such as SUBB), it becomes a horrible mess of 10 or 20 instructions to emulate a differing architecture.

So, the C language (I suspect, but I'm NOT sure of the actual reasons), by NOT going too low a level on purpose, avoids having to do lots of messing around and slowing things, because of all the extra instructions needed to fix the compatibility issues, between architectures.

As long as some method of putting in custom assembly code and/or intrinsics, is included in a new language.  It should not be necessary, to go too overboard, with low level functionality.  Because the programmers, can just put in assembly code or similar, as and where necessary.

I.e. C seems to stick to things, which is relatively common, on just about all architectures, rather than risk taking too many assumptions and/or having to do (potentially slow) fixes, to recreate the compatibility.

You raise some good points. But your example using a = a - 2; is something done routinely, even in C. You'd need to show a better example of an abstract idea that's "low level" and that C doesn't support, that you consider would be too low level.

C does support embedded assembly language though, can one get any lower level than that?

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 18, 2022, 12:27:53 pm
A few years ago I wrote a top octave generator using an Arduino nano (16MHz clock, ATMega328 CPU). It simultaneously produced 12 square waves on 12 pins with frequencies from 4435Hz to 8368Hz with transitions to the nearest 500ns. It would be interesting to see what processor could achieve that with an interpreter. Or even with a compiler. The object code was huge but in the main generated by assembler macros

So, you wrote it in assembly   :o

Some mpus have hw helpers.
Special programmable circuits that only need glue code.

my-c does not optimize the generated assembly code
It's a problem if you need performance.
But it's not its purpose.

What is "my-c"? as written it sounds like "mice" or maybe "Mike".
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 18, 2022, 12:38:13 pm
C does support embedded assembly language though, can one go any lower?
what kind of nonsense question is that? the plain answer for beginner is no. if yes, please elaborate. you may prove i'm a beginner as well! ;) or maybe we are using different dictionary here? if you keep posting nonsensical thing like this and before, dont be surprised this thread will become amusement center, which from experience, will eventually get locked down, you dont want it to happen do you?
Title: Re: A microcontroller programming language
Post by: DiTBho on December 18, 2022, 04:58:08 pm
What is "my-c"? as written it sounds like "mice" or maybe "Mike".

It's a C compiler I initially wrote to support a MIPS5+ prototype.
Later, I modified it for internal company needs.
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 18, 2022, 05:04:51 pm
I didn't explain the a = a - 2 section very well....
you keep rephrasing that same basic stuffs, even if you explained it in pseudo/general idea earlier, anybody with basic/intermediate knowledge should understand very well the point you want to make.. you forget he's an expert an experienced programmer, he already know that yet insist on continuing... let him do his stuffs. you cant win!
Title: Re: A microcontroller programming language
Post by: DiTBho on December 18, 2022, 05:21:17 pm
But the issue would be that if you included that statement in the language, and people start using it, even when it was not really needed.  Any architectures, that DON'T have a 64 bit, bit count instruction, may run that instruction very inefficiently.

it's always a compromise, hardware/profile can only mitigate this.

1: compatibility -> software emulated instructions when not hardware implemented. You know, it will be slower.
2: speed -> full use of the hardware, only if possible
3: nothing -> always refuse to compile if you are not willing to accept none of the above
4: deal -> always find a "sub solution" between the first two points

my-c encourages (actually forces) you to segregate critical parts into modules.
So you can easily think about what you have to do to support your specific hardware/profile

I mean, if you do that, say you have 10 .c files, 8 are OK, and will be compiled, 2 will be rejected.
You know you have to think about how to replace those 2 (even with modules written in assembly).

Talking about my job experience, segregated critical parts is also good practice for test-report.
(you save time, same result to pass the QA with less effort)
Title: Re: A microcontroller programming language
Post by: Mechatrommer on December 18, 2022, 05:32:14 pm
Although you are right, that some programmers, might have understood it.  It should have been written clearly enough so that all or nearly all, programmers, could understand it.
you cant win against nitpickers, and esp trolls (when they became apparent... since no better argument can be given)... requoted below..

C does support embedded assembly language though, can one go any lower?

it seems he is quite fixed, hence he doesnt need anymore "negative" advice. if you like something he post next, then you can talk ;) cheers.

ps: how could i've forgotten this button? (attached picture) :palm: thanks God!
and oh... this topic also in the list... could be a good bedtime reading...
https://www.eevblog.com/forum/programming/too-many-programming-languages/ (https://www.eevblog.com/forum/programming/too-many-programming-languages/)

cheers.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 18, 2022, 06:13:25 pm
You raise some good points. But your example using a = a - 2; is something done routinely, even in C. You'd need to show a better example of an abstract idea that's "low level" and that C doesn't support, that you consider would be too low level.

I didn't explain the a = a - 2 section very well.

Another (possibly better) example, would be a low level instruction, which counts how many bits in total, are set (1), in an unsigned, 64 bit integer.  Which is useful for some types of algorithms and program types.

So the high level language, could include a low level statement, which counts up the bits set count, works in only 1 CPU clock cycle (because the initial CPU architecture has such an assembly language instruction), for the current CPU architecture, and so runs very quickly.

But the issue would be that if you included that statement in the language, and people start using it, even when it was not really needed.  Any architectures, that DON'T have a 64 bit, bit count instruction, may run that instruction very inefficiently.
There are faster methods, but essentially the function would be performing something like:

// Code vaguely based on Pascal
// Count the total bits set in variable x
bit_count = 0;
for bit = 0 to 63 do
begin
if x and 1 == 1 then bit_count++;
x /= 2;
end

There are faster algorithms.

C does support embedded assembly language though, can one get any lower level than that?

That is an interesting question.  Computers are getting rather complicated these days, because as well as the CPU which processes conventional instructions.  There can be various other sections (System on a chip, SoC), such as AI/Machine learning accelerators (technical names can vary), hardware encryption, video encoders/decoders.  Even finite state machines (FSMs), FPGAs, the list these days, seems to go on and on.

So I suppose it depends on ones definition.  But anyway, technically speaking some of these SoC (names can vary), features, can be at a relatively simple/low level (lower than standard double floating point or 64 bit integers), but typically, it is done at incredibly high speeds.  Compared to the main/normal/standard CPU instruction set.

OK I see, that's a better example. Yes the bit count thing is something I was reading about recently, they refer to it as "popcount". The concern then is, that a person might write code using that or copy/paste code using that and have no inkling that there might be a much higher cost than they would have thought.

So this raises the question of intrinsic operations, how to deal with and implement these, its a good question, a very relevant issue, I agree.

Here's a Microsoft article (https://learn.microsoft.com/en-us/cpp/intrinsics/popcnt16-popcnt-popcnt64?view=msvc-170) on this very issue of popcount.

Note:

Quote
If you run code that uses these intrinsics on hardware that doesn't support the popcnt instruction, the results are unpredictable.

You've raised a pretty good issue here, the very kind of thing that must be carefully thought about in a new language. It's an interesting area, there are no reserved words too so naming and adding intrinsic over time can't break existing code, this is a good example of why having no reserved words is a help, I know I may have seemed to dwell on this a lot, but if one doesn't design the grammar to do that there's no going back, incorporating this gives a great deal of power.

Now PL/I (and this new language - potential new language...) has what are called "builtin" functions. These are functions inherent in the language and they way they did that is clever.

To use a builtin function you MUST declare the function, declaring is very easy too, whatever the function is, whatever it returns and whatever arguments it takes, it is always declared as:

Code: [Select]

dcl Substr builtin;
dcl Sqrt builtin;
dcl Offset builtin;


Of course a bunch of those could easily be inside some utility include file. The attribute 'builtin' is sufficient, since the compiler knows already about the function's signature. This is how PL/I represents the kind of things we see in C with "stdlib.h" but is much slicker I think.

Now because keywords are not reserved, a user might create a function named 'Sqrt' and that's fine and legal, because unless they've already declared it as builtin, there's no issue, if they later wanted to use the real builtin one, they realize the conflict and resolve it. This means that adding new builtins also cannot break existing code, unless a builtin is explicitly declared it does no harm and a user cannot declare a function as builtin if its not yet present in the version of the language they're using.

Anyway, I'm thinking a similar idea could well be applied to intrinsics, naturally we'd consider something like:

Code: [Select]

dcl Popcount intrinsic;
dcl Emit intrinsic;
dcl Prefetch intrinsic;


Then during compilation we'd issue a diagnostic if the intrinsic was not available for the target CPU or if it is available but might be costly...

Title: Re: A microcontroller programming language
Post by: gnuarm on December 18, 2022, 06:30:44 pm
Long thread, so I have not read the entire thing.  I will say that anytime you are working close to the metal, or even just controlling things, Forth is an excellent choice.  Yet, I haven't seen it mentioned very much here. 

Forth is my go to language for any programming I do outside of a spreadsheet. 

Why are people not more interested in Forth?  Is it just a matter of it being so different from languages like C?
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 18, 2022, 06:32:33 pm
Long thread, so I have not read the entire thing.  I will say that anytime you are working close to the metal, or even just controlling things, Forth is an excellent choice.  Yet, I haven't seen it mentioned very much here. 

Forth is my go to language for any programming I do outside of a spreadsheet. 

Why are people not more interested in Forth?  Is it just a matter of it being so different from languages like C?

I'm interested in many languages, I admit to knowing very little about Forth, I read a lot about it years ago, it was used early on when MCU's came out in the 1970s.

So tell me about Forth, what do you like about it?

Title: Re: A microcontroller programming language
Post by: tggzzz on December 18, 2022, 07:13:41 pm

C does support embedded assembly language though, can one get any lower level than that?

Yes, you can.

Current Intel processors like this: https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/secure-coding/loading-microcode-os.html (https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/secure-coding/loading-microcode-os.html)

Current AMD processors similarly.

With previous Intel processors it was more interesting and explicit: see their 3000 series processors. For the superior AMD equivalent, see their 2900 series.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 18, 2022, 07:33:04 pm
It's why POWER10 is publicized as "safe".
IBM RISC, doesn't allow any microcode.
It has no microcode.

Assembly is the lowest level you can go.
And it mostly looks ~ppc64
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 18, 2022, 07:38:14 pm
Microcode upload is one spectacularly great security hole. But it allows CPU vendors to rush new models into production.
Title: Re: A microcontroller programming language
Post by: gnuarm on December 18, 2022, 08:11:45 pm
Long thread, so I have not read the entire thing.  I will say that anytime you are working close to the metal, or even just controlling things, Forth is an excellent choice.  Yet, I haven't seen it mentioned very much here. 

Forth is my go to language for any programming I do outside of a spreadsheet. 

Why are people not more interested in Forth?  Is it just a matter of it being so different from languages like C?

I'm interested in many languages, I admit to knowing very little about Forth, I read a lot about it years ago, it was used early on when MCU's came out in the 1970s.

So tell me about Forth, what do you like about it?

Forth is interactive, you type on the command line and it provides immediate results.  Subroutines are called word definitions.  Words can be tested from the command line.  This lets you test your work coded from the bottom up. 

The Forth tool itself is very small.  You can implement it on an MCU using just 8 kB to 16 kB of program store and some small amount of RAM.  This means your development system is the target MCU itself!  Your computer is a terminal and the mass storage.  Compiling a program is just a matter of dumping it through the serial port, or Ethernet port, or USB port the target is connected to. 

The bottom line is, it's very flexible. 

I am going to be rewriting a test fixture control program written in Forth, to handle multiple test units simultaneously.  I'm looking forward to this.
Title: Re: A microcontroller programming language
Post by: brucehoult on December 18, 2022, 08:17:57 pm
Long thread, so I have not read the entire thing.  I will say that anytime you are working close to the metal, or even just controlling things, Forth is an excellent choice.  Yet, I haven't seen it mentioned very much here. 

Forth is my go to language for any programming I do outside of a spreadsheet. 

Why are people not more interested in Forth?  Is it just a matter of it being so different from languages like C?

Forth has weird conventions and mechanisms.

I prefer PostScript. It is much improved over Forth, clearer to read, better data structures, and more dynamic by default, though you can "bind" things for speed when needed.
Title: Re: A microcontroller programming language
Post by: gnuarm on December 18, 2022, 10:41:28 pm
Why are people not more interested in Forth?  Is it just a matter of it being so different from languages like C?

Forth has weird conventions and mechanisms.

I prefer PostScript. It is much improved over Forth, clearer to read, better data structures, and more dynamic by default, though you can "bind" things for speed when needed.

What weird conventions and mechanisms?
Title: Re: A microcontroller programming language
Post by: gnuarm on December 18, 2022, 10:45:02 pm
Forth has weird conventions and mechanisms.

I prefer PostScript. It is much improved over Forth, clearer to read, better data structures, and more dynamic by default, though you can "bind" things for speed when needed.

sectorforth, see here:
https://github.com/cesarblum/sectorforth

Fits the entire Forth system into a tiny, single 512 byte sector (of a floppy disk, I think).  Which shows the power of both assembly language it is mainly written in, Forth, and some peoples determination, to do things, just because they can.

That is so small, you could fit it into just 8 of the 16 or 32 registers of an AVX-512 equipped processor.  Call it just 25% of the AVX-512 accumulators/registers, or an even lower percentage, if you include the mask registers.

How long would it take, to create a Postscript system, complete with all its 400+ commands/instructions (an average of a byte or so, for each one), to fit into a single 512 byte sector?

I think for some new programming languages.  Forth's potential, high efficiency in some areas, and the fact that much of it, is actually written in Forth itself, makes it interesting.  Because it allows final end users (programmers), to freely extend its functionality, as if it was part of the original interpreter.

But as suggested in quote, PostScript, seems interesting to consider as well.  Thanks for mentioning it.  I seem to very strongly associate it with Laser Printers, rather than seriously considering it as a programming language, for various reasons.  Also, I consider GhostScript, a rather unreliable and buggy way of trying the language out.  At least it (GhostScript) was (rather/very buggy), a long time ago.

512 bytes is very small, even for a Forth.  Are there no additional word definitions that get loaded as source?  Source compilation is so fast, that systems often have just a core in binary, and load the remainder from source. 

I don't know of any PostScript tools that are generally available for embedded systems.  Would using PS on an ARM with 16 kB of Flash even be possible?
Title: Re: A microcontroller programming language
Post by: cfbsoftware on December 18, 2022, 11:04:45 pm
C does support embedded assembly language though, can one get any lower level than that?
One of the features of softcore FPGA CPUs is that you can implement specialised operations using bit patterns which were previously undefined in the instruction set. These bit patterns can be emitted directly by the compiler without using an assembler.     
Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 12:16:05 am
Quote
These bit patterns can be emitted directly by the compiler without using an assembler.     

emitted how? By inline asm DC?
If the machine layer is not awere of them and they are emmited as "raw" data, then it's a trick

Sure it works, but it's an hack
Does not pass QA
Makes ICE confused
Makes the source dirty
Makes the compiler unable to understand what's going on
Makes the source not portable
Opens doors to undefined run time behaviors
Title: Re: A microcontroller programming language
Post by: gnuarm on December 19, 2022, 12:34:35 am
512 bytes is very small, even for a Forth.  Are there no additional word definitions that get loaded as source?  Source compilation is so fast, that systems often have just a core in binary, and load the remainder from source.

I've wanted to try it out for some time, but been put off, because you have to sort out some kind of 386 system (e.g. an emulator), to try it out.

Looking at the hello world, example program.  Here:
https://github.com/cesarblum/sectorforth/blob/master/examples/01-helloworld.f

This is what I expected.  If the application has to define words like branch and ?branch and lit, then this is a very primitive kernel.  Not many systems are this primitive, because it requires so much to be included in the application.  But if you only have 512 bytes!

Compiling in Forth is unusually fast, and only single pass.  The dictionary structure allows words to be compiled without spending lots of time resolving references, etc. 


Quote
It seems there are only 8 primitive words (see readme file for more information, sorry, I called them functions in my earlier post, I tend to call things functions or procedures, or even subroutines, even if the language prefers other names).  So the hello world example program, defines a very large number of relatively primitive functions words, in order to make the required program.

In fairness, 512 bytes is rather tight to fit a full (ALL) Forth word, system into.  But even so, to have enough of a Forth System, to make a usable entity, is still very impressive!

E.g. The hello world program has definitions, such as:

Quote
\ stack manipulation words
: drop ( x y -- x ) dup - + ;
: over ( x y -- x y x ) sp@ 2 + @ ;
: swap ( x y -- y x ) over over sp@ 6 + ! sp@ 2 + ! ;
: nip ( x y -- y ) swap drop ;
: 2dup ( x y -- x y x y ) over over ;
: 2drop ( x y -- ) drop drop ;

So it is sort of cheating.  But it also shows the expressive power of Forth, rather nicely.

It's not exactly cheating.  Some C implementations will only include parts of libraries that are actually invoked, to keep the executable small.  In this kernel, it's up to the user to figure out what to include and what to ignore.  Sometimes that lets you do things, you otherwise would not be able to pull off.  In the general case, it's a PITA. 

Of course, it's easy enough to work around.  I have some of my own libraries I simply include in their entirety, because I'm working on a PC where my programs could fit in the level 1 cache. 
Title: Re: A microcontroller programming language
Post by: gnuarm on December 19, 2022, 12:39:53 am
Quote
These bit patterns can be emitted directly by the compiler without using an assembler.     

emitted how? By inline asm DC?
If the machine layer is not awere of them and they are emmited as "raw" data, then it's a trick

Sure it works, but it's an hack

Calling it a trick or a hack is silly.  I designed a bit slice processor as a DMA controller for an array processor (the supercomputer of the 70s).  I wanted to use some instructions that I created, which were not part of the assembler by default.  I added them.  QA didn't complain.  I just had to document it.


Quote
Does not pass QA
Makes ICE confused
Makes the source dirty
Makes the compiler unable to understand what's going on
Makes the source not portable
Opens doors to undefined run time behaviors

ICE???  If you need an ICE, your program is far too complex. 

You use a lot of nasty sounding adjectives.  Where have you been working, porn web sites?

If Forth is your compiler, it is entirely extensible, so there's no problem with it not "understanding". 
Title: Re: A microcontroller programming language
Post by: brucehoult on December 19, 2022, 01:41:20 am
How long would it take, to create a Postscript system, complete with all its 400+ commands/instructions (an average of a byte or so, for each one), to fit into a single 512 byte sector?

I'm not suggesting to implement the graphics library! Just the programming language.

abs add aload and astore array begin bind cleartomark copy count cvi cvlit cvn cvs cvx def dict idiv dup end eq exch exit false for forall ge get gt if ifelse index known le length load loop lt mark mod mul ne neg or pop put repeat restore roll save string store sub true where xor { } [ ]

That should be a pretty good start. Could even leave some stuff out. I count 61 operators there. I'm assuming PostScript Level 1 memory management here, with mark/release style dynamic memory allocation (called save/restore in PostScript).

Quote
Also, I consider GhostScript, a rather unreliable and buggy way of trying the language out.  At least it (GhostScript) was (rather/very buggy), a long time ago.

GhostScript is used EVERYWHERE in the last 20 years. Sure, maybe it was buggy in 1988, but these days it's in the workflow of everyone's dot matrix printer drivers, millions of web sites converting various document types to and from PDF, and on and on.

No, it's not buggy, any more than any other software.
Title: Re: A microcontroller programming language
Post by: brucehoult on December 19, 2022, 01:44:53 am
I don't know of any PostScript tools that are generally available for embedded systems.  Would using PS on an ARM with 16 kB of Flash even be possible?

Of course. Why not?  It's very similar in complexity to Forth, and much simpler than, say, Python or JavaScript, both of which have small microcontroller implementations.
Title: Re: A microcontroller programming language
Post by: gnuarm on December 19, 2022, 02:25:27 am
I don't know of any PostScript tools that are generally available for embedded systems.  Would using PS on an ARM with 16 kB of Flash even be possible?

Of course. Why not?  It's very similar in complexity to Forth, and much simpler than, say, Python or JavaScript, both of which have small microcontroller implementations.

If it's like Forth, then how is it different, better?

LOL!  Don't even compare Python to Forth. 
Title: Re: A microcontroller programming language
Post by: gnuarm on December 19, 2022, 02:28:48 am
Quote
PostScript works best in a non-interactive batch processing mode.

This is the exact opposite of Forth... well, I shouldn't say opposite.  Forth handles batch mode just fine.  But the interactivity of Forth is a powerful debugging technique. 
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 02:34:09 am
Probably the most fundamental criteria for a systems programming language to meet, is, "Can I write an operating system with it?".

This separates the men from the boys so to speak, if we look at historical delivered/shipped operating systems, then its a relatively short list.

So the primary goal of a new language, well suited for MCU development, is it be capable of serving as the optimal high level language just above the native machine's assembler, the ideal language for writing an operating system even, since many scenarios won't need or desire an OS to support them and must be able to implement core services directly.

What features would we insist upon for such a lofty goal? What are the non-negotiable features we'd need, for a language we'd rely on for writing an operating system on any modern MPU/MCU?



Title: Re: A microcontroller programming language
Post by: gnuarm on December 19, 2022, 03:58:56 am
There are several different domains for operating systems, each very different.  The sort of OS that runs on a PC type computer is a totally different OS from what runs on a small MCU, if any OS at all! 

The sort of work I do doesn't even use an MCU for the most part.  I like parallelism, and that's just too much work on an MCU, so I use an inherently parallel language to write code that runs on the many processors available in FPGAs.  I'm not talking about CPU designs, I mean, the fact that every piece of logic is inherently a process.  Since they work in parallel, they are parallel processes. 

In VHDL this code is multiple processes.

Code: [Select]
  MCLK_Enable <= ( MCLK_Divide = 0 );
  BCLK_Enable <= ( BCLK_Divide = 0 );
  LRCK_Enable <= ( LRCK_Divide = 0 );

No OS required. 

In Forth the above could be...

Code: [Select]
: test ( MCLK_Divide, BCLK_Divide, LRCK_Divide -- MCLK_Enable, BCLK_Enable, LRCK_Enable)
  0= rot 0= rot 0= rot ;

Or maybe...

Code: [Select]
MCLK_Divide 0= rot DoSomethingMCLKWithTheResult
BCLK_Divide 0= rot DoSomethingBCLKWithTheResult
LRCK_Divide 0= rot DoSomethingLRCKWithTheResult

Not parallel processes.  So if timing of these calculations is important, you need a mechanism to synchronize with the timing cue.  That's in DoSomething...WithTheResult, etc.  But still no OS. 

Isn't hardware just so much easier?
Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 04:16:58 am
Quote

ICE???  If you need an ICE, your program is far too complex. 

You use a lot of nasty sounding adjectives.  Where have you been working, porn web sites?

 :-DD
Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 04:28:42 am
Quote
Calling it a trick or a hack is silly.  I designed a bit slice processor as a DMA controller for an array processor (the supercomputer of the 70s).  I wanted to use some instructions that I created, which were not part of the assembler by default.  I added them.  QA didn't complain.  I just had to document it.

Whatever, emitted how, so?
If it's inline asm DC. Then it's an hack.

Quote
QA didn't complain.  I just had to document it.

Doc is good but doesn't pay money.
Code rejected means no money.
Title: Re: A microcontroller programming language
Post by: brucehoult on December 19, 2022, 04:38:37 am
Quote
PostScript works best in a non-interactive batch processing mode.

This is the exact opposite of Forth... well, I shouldn't say opposite.  Forth handles batch mode just fine.  But the interactivity of Forth is a powerful debugging technique.

EVERY programming language works better in non-interactive mode for large and complex programs.
 
It's not the opposite of Forth, it's similar to Forth except BETTER for interactive programming and debugging, because Postscript is late-binding by default. This means that you can define your words in any order, whether the words they use have been defined already or not, and you can re-define a word while debugging and existing words will automatically use the new definition.

This is not possible (or at least is not usually done) in address-threaded or subroutine-threaded Forth. It is more easily done in token-threaded Forth (the slowest implementation).

When speed is needed, individual functions or even bodies of IFs or loops can be early-bound (address-threaded, essentially)
Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 04:40:48 am
The first kernel of riscos was written in arm-assembly, but with smart macros

The first kernel of amigaos was written in m68k-macro-assembly, and somehow looked like "C--"

Linux is written in C but emulating some OO capabilities, somehow looks "C+" (not "C++", but emulating some of its features)

The first word Linux driver written in Rust for apple m1 is interesting

Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 19, 2022, 04:47:42 am
The first word Linux driver written in Rust for apple m1 is interesting

Have you looked at it?
Title: Re: A microcontroller programming language
Post by: gnuarm on December 19, 2022, 05:23:12 am
Quote
Calling it a trick or a hack is silly.  I designed a bit slice processor as a DMA controller for an array processor (the supercomputer of the 70s).  I wanted to use some instructions that I created, which were not part of the assembler by default.  I added them.  QA didn't complain.  I just had to document it.

Whatever, emitted how, so?
If it's inline asm DC. Then it's an hack.

I don't recall, that was nearly 50 years ago.  I seem to recall one supervisor being initially upset that technically, this used a different assembler, so it must not have been a DC, rather a "hack" to the assembler code. 

But once it was thoroughly documented in the project, he was happy.

You should never learn Forth coding.  I am confident it would blow your mind... and not in the good way.  Some people aren't bound so rigidly.


Quote
Quote
QA didn't complain.  I just had to document it.

Doc is good but doesn't pay money.
Code rejected means no money.

Sorry, you are incomprehensible.  Can you speak English?
Title: Re: A microcontroller programming language
Post by: gnuarm on December 19, 2022, 05:49:15 am
Quote
PostScript works best in a non-interactive batch processing mode.

This is the exact opposite of Forth... well, I shouldn't say opposite.  Forth handles batch mode just fine.  But the interactivity of Forth is a powerful debugging technique.

EVERY programming language works better in non-interactive mode for large and complex programs.

That's very bogus.  Any large program is developed in pieces, which are defined in pieces, etc., until they constitute small programs. 

When people run an ICE on a project, they are using the same methods used interactively, but with much more complexity because of the massive code base.

Forth is actually a bit famous for simplifying most coding.  Once you learn the philosophy, it becomes second nature to work in the "Forth" way testing the lower level code thoroughly before integrating at the next level. 

But people who are entrenched with using tools like ICE, don't appreciate, or possibly understand simpler ways of working.


Quote
It's not the opposite of Forth, it's similar to Forth except BETTER for interactive programming and debugging, because Postscript is late-binding by default.

I was responding to this quote from the web page...

"PostScript works best in a non-interactive batch processing mode."

Now you are saying Postscript is interactive.  Someone needs to make up their minds.


Quote
This means that you can define your words in any order, whether the words they use have been defined already or not, and you can re-define a word while debugging and existing words will automatically use the new definition.

I'm no expert on compilation, but it sounds to me that late-binding would be a bit slow.


Quote
This is not possible (or at least is not usually done) in address-threaded or subroutine-threaded Forth. It is more easily done in token-threaded Forth (the slowest implementation).

When speed is needed, individual functions or even bodies of IFs or loops can be early-bound (address-threaded, essentially)

Actually, Forth supports something similar, using words DEFER and IS.  This must be set up at compile time, but can be changed at run time.  So interactive debugging with alternate definitions for a given word is supported.  There is no run time penalty on most systems, while on others it would be slight.  It is essentially like changing the subroutine call by using an intermediary. 

I've used this for changing an interface between a serial port or a socket connection.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 05:51:08 am
The first word Linux driver written in Rust for apple m1 is interesting

Have you looked at it?

Yup. I looked at it in very superficial ways, I have no specific RUST knowledge to make judgments.
I will  :-//
Title: Re: A microcontroller programming language
Post by: brucehoult on December 19, 2022, 06:01:38 am
EVERY programming language works better in non-interactive mode for large and complex programs.

That's very bogus.  Any large program is developed in pieces, which are defined in pieces, etc., until they constitute small programs. 

When people run an ICE on a project, they are using the same methods used interactively, but with much more complexity because of the massive code base.

Forth is actually a bit famous for simplifying most coding.  Once you learn the philosophy, it becomes second nature to work in the "Forth" way testing the lower level code thoroughly before integrating at the next level. 

But people who are entrenched with using tools like ICE, don't appreciate, or possibly understand simpler ways of working.

I wouldn't know an ICE if it bit me in the arse.

BASIC is interactive. Python is interactive. Both are better written in a text editor, except in rare circumstances where you don't know WTF the the language does or want to poke at some peripheral.

Quote
Quote
It's not the opposite of Forth, it's similar to Forth except BETTER for interactive programming and debugging, because Postscript is late-binding by default.

I was responding to this quote from the web page...

"PostScript works best in a non-interactive batch processing mode."

Now you are saying Postscript is interactive.  Someone needs to make up their minds.

Dude, that's not my web page. It's the opinion of someone who is not in this discussion.

Something being interactive doesn't mean you want to work that way all the time.

Quote
Quote
This means that you can define your words in any order, whether the words they use have been defined already or not, and you can re-define a word while debugging and existing words will automatically use the new definition.

I'm no expert on compilation, but it sounds to me that late-binding would be a bit slow.

OF COURSE it is. It's also far more flexible. That's why lots and lots of people program most of their application logic in slow-as-molasses Python, and do the few bits where you can tell the difference in C.

It's better if you can do it all in one language.

Quote
Quote
This is not possible (or at least is not usually done) in address-threaded or subroutine-threaded Forth. It is more easily done in token-threaded Forth (the slowest implementation).

When speed is needed, individual functions or even bodies of IFs or loops can be early-bound (address-threaded, essentially)

Actually, Forth supports something similar, using words DEFER and IS.  This must be set up at compile time, but can be changed at run time.  So interactive debugging with alternate definitions for a given word is supported.  There is no run time penalty on most systems, while on others it would be slight.  It is essentially like changing the subroutine call by using an intermediary. 

Which means Forth has is bass-ackwards, as the flexible slow method should be the DEFAULT. not something you have to jump through hoops to get.

I've been using Forth, on and off for more than 40 years, when appropriate. And PostScript for 37 years.
Title: Re: A microcontroller programming language
Post by: cfbsoftware on December 19, 2022, 06:02:57 am
Quote
These bit patterns can be emitted directly by the compiler without using an assembler.     
emitted how? By inline asm DC?
No. All of the code generated by our Oberon compilers (as is true for most, if not all, of Prof Wirth's compilers based on) is written as 16-bit or 32-bit words directly into the binary file. No assembly code is generated; no assembler pass is necessary or is used.

e.g. for the Project Oberon RISC5 FPGA compiler:

Quote
Instructions are emitted sequentially and emitted by the four procedures Put0, Put1, Put2, Put3. They directly correspond to the instruction formats of the RISC processor (see Chapter 11). The instructions are stored in the array code and the compiler variable pc serves as running index.

https://people.inf.ethz.ch/wirth/ProjectOberon/PO.Applications.pdf


Title: Re: A microcontroller programming language
Post by: gnuarm on December 19, 2022, 06:35:59 am
EVERY programming language works better in non-interactive mode for large and complex programs.

That's very bogus.  Any large program is developed in pieces, which are defined in pieces, etc., until they constitute small programs. 

When people run an ICE on a project, they are using the same methods used interactively, but with much more complexity because of the massive code base.

Forth is actually a bit famous for simplifying most coding.  Once you learn the philosophy, it becomes second nature to work in the "Forth" way testing the lower level code thoroughly before integrating at the next level. 

But people who are entrenched with using tools like ICE, don't appreciate, or possibly understand simpler ways of working.

I wouldn't know an ICE if it bit me in the arse.

BASIC is interactive. Python is interactive. Both are better written in a text editor, except in rare circumstances where you don't know WTF the the language does or want to poke at some peripheral.

I think you fail to understand.  No one writes Forth code at the console, then writing it again into an editor.  But when you have unexpected results, you can retype the definition in question and explore what it does... interactively.  Interactive debugging. 

Interactive is a good way to first pass test the code by running it with corner cases of data input.  In fact, many Forth users include the test code with their source so it can be tested at any time.


Quote
Quote
Quote
It's not the opposite of Forth, it's similar to Forth except BETTER for interactive programming and debugging, because Postscript is late-binding by default.

I was responding to this quote from the web page...

"PostScript works best in a non-interactive batch processing mode."

Now you are saying Postscript is interactive.  Someone needs to make up their minds.

Dude, that's not my web page. It's the opinion of someone who is not in this discussion.

Fine, but it was posted.  If you didn't post it, why are you responding???


Quote
Something being interactive doesn't mean you want to work that way all the time.

Who said you have to work interactively all the time? 

You seem very jumpy about this.  What is bothering you?


Quote
Quote
Quote
This means that you can define your words in any order, whether the words they use have been defined already or not, and you can re-define a word while debugging and existing words will automatically use the new definition.

I'm no expert on compilation, but it sounds to me that late-binding would be a bit slow.

OF COURSE it is. It's also far more flexible. That's why lots and lots of people program most of their application logic in slow-as-molasses Python, and do the few bits where you can tell the difference in C.

It's better if you can do it all in one language.

Quote
Quote
This is not possible (or at least is not usually done) in address-threaded or subroutine-threaded Forth. It is more easily done in token-threaded Forth (the slowest implementation).

When speed is needed, individual functions or even bodies of IFs or loops can be early-bound (address-threaded, essentially)

Actually, Forth supports something similar, using words DEFER and IS.  This must be set up at compile time, but can be changed at run time.  So interactive debugging with alternate definitions for a given word is supported.  There is no run time penalty on most systems, while on others it would be slight.  It is essentially like changing the subroutine call by using an intermediary. 

Which means Forth has is bass-ackwards, as the flexible slow method should be the DEFAULT. not something you have to jump through hoops to get.

I've been using Forth, on and off for more than 40 years, when appropriate. And PostScript for 37 years.

Yeah, you have a bug up your rear about this.  I would like to discuss this with you, but you are just so argumentative. 

You have your opinions (very strong ones) and I have mine.  Thanks for sharing.
Title: Re: A microcontroller programming language
Post by: Kalvin on December 19, 2022, 09:19:52 am
Here are my 2c (Did not read through the whole thread, so please forgive me if these are dupilcates):

1. In C the arrays are quite problematic when passed to functions, because the length of the array is not implicitly passed as an argument. So, the new language should pass the length of an array implicitly to the function to be called. The length of the passed array needs to be accessible inside the called function. Enabling/disabling run-time checks for validating array accesses should be supported globally / by compilation unit / by function, and during unit testing. Also, when passing a variable or an array as a void*, the caller should pass the total size of the item in bytes to the called function.

2. In C handling of endianess when declaring structs and bit fields required some work in order to make code portable across different architectures: Keywords and language constructs for native / big endian / little endian should be provided, and the compiler should take care of producing code for the correct endianess.

3. In C the padding of structs requires careful design to get it right, especially when porting to different architectures. The compiler should provide simple and intuitive means for specifying and asserting the padding.

4. Design by contract (https://en.wikipedia.org/wiki/Design_by_contract) should be supported natively by the compiler so that the verification of the contracts can be enabled during compilation time and when running the unit tests. The run-time validation of the contract asserts, pre- and postcondition contracts should be controllable globally / by compilation unit / by function when creating for the actual production build. In C this can be done using macros, but this should be built-in into the language, and should be supported during run-time system.

5. Modern C compilers complain when comparing unsigned value to a signed value, which is good. But modern C compilers allow assigning unsigned variables to signed variables, and vice versa. Assigning unsigned variables to signed variables, and vice versa, should be flagged as an error by default in general. Literal values should be casted automatically by the compiler, of course. If the user wants to assign signed variables to unsigned variables, and vice versa, the operation should be done explicitly by the programmer.

6. Native 8-bit unsigned and signed data types, and a separate 8-bit char data type which is not compatible with the integer types without proper casting.

7. Strict type checking. For example the following should not be allowed without explicit cast, because they are declared to be different data types although they are both int8_t types at the end of the day:

Code: [Select]
typedef int8_t apple_t;
typedef int8_t orange_t;

apple_t apples = 0;
orange_t oranges = 0;

apples = oranges; /* This should be flagged as an error without explicit cast */

Edit:
8. More intuitive way to work with bits/bitmasks.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 11:11:53 am
Here are my 2c (Did not read through the whole thread, so please forgive me if these are dupilcates):

1. In C the arrays are quite problematic when passed to functions, because the length of the array is not implicitly passed as an argument. So, the new language should pass the length of an array implicitly to the function to be called. The length of the passed array needs to be accessible inside the called function. Enabling/disabling run-time checks for validating array accesses should be supported globally / by compilation unit / by function, and during unit testing. Also, when passing a variable or an array as a void*, the caller should pass the total size of the item in bytes to the called function.

2. In C handling of endianess when declaring structs and bit fields required some work in order to make code portable across different architectures: Keywords and language constructs for native / big endian / little endian should be provided, and the compiler should take care of producing code for the correct endianess.

3. In C the padding of structs requires careful design to get it right, especially when porting to different architectures. The compiler should provide simple and intuitive means for specifying and asserting the padding.

4. Design by contract (https://en.wikipedia.org/wiki/Design_by_contract) should be supported natively by the compiler so that the verification of the contracts can be enabled during compilation time and when running the unit tests. The run-time validation of the contract asserts, pre- and postcondition contracts should be controllable globally / by compilation unit / by function when creating for the actual production build. In C this can be done using macros, but this should be built-in into the language, and should be supported during run-time system.

5. Modern C compilers complain when comparing unsigned value to a signed value, which is good. But modern C compilers allow assigning unsigned variables to signed variables, and vice versa. Assigning unsigned variables to signed variables, and vice versa, should be flagged as an error by default in general. Literal values should be casted automatically by the compiler, of course. If the user wants to assign signed variables to unsigned variables, and vice versa, the operation should be done explicitly by the programmer.

6. Native 8-bit unsigned and signed data types, and a separate 8-bit char data type which is not compatible with the integer types without proper casting.

7. Strict type checking. For example the following should not be allowed without explicit cast, because they are declared to be different data types although they are both int8_t types at the end of the day:

Code: [Select]
typedef int8_t apple_t;
typedef int8_t orange_t;

apple_t apples = 0;
orange_t oranges = 0;

apples = oranges; /* This should be flagged as an error without explicit cast */

Edit:
8. More intuitive way to work with bits/bitmasks.

This is an excellent post, very helpful. I will respond later, many of the points you raise were already being considered but you've include several insightful additions.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 12:34:30 pm
I don't recall, that was nearly 50 years ago.  I seem to recall one supervisor being initially upset that technically, this used a different assembler, so it must not have been a DC, rather a "hack" to the assembler code. 

No way, if it was a C file with assembly inline DC, it was an hack!

my-c can fix this, but it requires an advanced set of internal mechanisms to define new operators that map to new hw opcodes in a consistent way to let the compiler know their boundaries, their negative side effect (if any), provide ways to observe and verify (this is useful for ICE test cases) and also provide code rejection on error.

[] -> { ok, maybe, nothing }

But once it was thoroughly documented in the project, he was happy.

documenting hacks, well ... yes, bettern than nothing

You should never learn Forth coding.  I am confident it would blow your mind... and not in the good way.  Some people aren't bound so rigidly.

I have ZERO interest in Forth. It's not even close to what I think.
I make programs in C and Ada for a living.
Recently I am looking at RUST with interest.

Sorry, you are incomprehensible.  Can you speak English?

You said that QA doesn't complain (when) you just have to document your code.
I said that doc is always a good practice but doesn't pay money.

to pay : give someone something in return for { work done, goods received, debt incurred, favor done }

pay(someone, something) ->
{
work done: your clients(code commited, QA passed) -> return you money
favor done: your colleagues(code documented, hacks demystified) -> return you gratitude
}

Documenting things is always a great thing, it helps you with future work on the same code, it repays your colleagues gratitude in return (if they have to work on your code), but what only matters to you at the end of the week is whether the code passes or fails QA.

My customers won't pay a cent for the document, and won't pay a cent if the code fails QA.

Thus, documenting has a lower priority.
It's not good, but it's how it works.
Title: Re: A microcontroller programming language
Post by: Kalvin on December 19, 2022, 12:41:01 pm
One final note: Classical if - then - elseif - else probelm. My suggestion is something like this:

Code: [Select]
if (expr)
  stmts;
elif (expr)
  stmts;
else
  stmts;
endif

Thus a simple test:

Code: [Select]
if (expr)
  stmts;
endif

A simple test with else clause:

Code: [Select]
if (expr)
  stmts;
else
  stmts;
endif

etc.
Title: Re: A microcontroller programming language
Post by: newbrain on December 19, 2022, 12:51:43 pm
7. Strict type checking. For example the following should not be allowed without explicit cast, because they are declared to be different data types although they are both int8_t types at the end of the day:

Code: [Select]
typedef int8_t apple_t;
typedef int8_t orange_t;

apple_t apples = 0;
orange_t oranges = 0;

apples = oranges; /* This should be flagged as an error without explicit cast */

Such a strict type checking might be good, or not. Opinions differ.
Not even Pascal does what you are suggesting.

Would you also object to apples = apples + 1; ?
If not, why not? It's a new type (casting away the fact that typedef in C does not introduce new types, only synonyms: C11 6.7.8 §3).

Shouldn't it be apples = (apple_t)((int8_t)apples + 1);?

After all, apple_t and int8_t are no longer the same type (if they were, so would be orange_t and we are back to C semantics), so why should I expect to have the same operations defined on them.

BTW: what would be the if-then-else if-else problem?
Title: Re: A microcontroller programming language
Post by: Kalvin on December 19, 2022, 01:03:04 pm
Would you also object to apples = apples + 1; ?
If not, why not? It's a new type (casting away the fact that typedef in C does not introduce new types, only synonyms: C11 6.7.8 §3).

Shouldn't it be apples = (apple_t)((int8_t)apples + 1);?

After all, apple_t and int8_t are no longer the same type (if they were, so would be orange_t and we are back to C semantics), so why should I expect to have the same operations defined on them.

The assignment and mixed arithmetic between different types should not be allowed by default.

These are just fine:
Code: [Select]
apples = apples + 1;
apples++;
apples = (apples << 1) | 1;

However, mixed arithmetic between different types should not be allowed by default:
Code: [Select]
apples = 1 + oranges;
apples = (oranges << 1) | 1;
apples = apples + oranges * 2

BTW: what would be the if-then-else if-else problem?

The classical dangling else problem, and accidental missing braces in C.

Edit:
typedef in C does not introduce new types, only synonyms: C11 6.7.8 §3)
I would like to see that typedef defines a new type. For declaring a synonym, a new keyword alias** for example, should be used instead.
** alias may not be a good keyword here, but you get the idea.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 01:06:32 pm
4. Design by contract

my-c does exactly that with both language-specific support and ICE. I'm very happy that someone finally talked about it ;D

it is very useful during the develpment phases, great later during test cases for test-reports, which then my customers are very happy to pay for, not only because it certifies (with reproducible procedures) QA passed, but also because they can easily re-use most of the code to create production "diagnostics".
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 01:19:21 pm
Here are my 2c (Did not read through the whole thread, so please forgive me if these are dupilcates):

1. In C the arrays are quite problematic when passed to functions, because the length of the array is not implicitly passed as an argument. So, the new language should pass the length of an array implicitly to the function to be called. The length of the passed array needs to be accessible inside the called function. Enabling/disabling run-time checks for validating array accesses should be supported globally / by compilation unit / by function, and during unit testing. Also, when passing a variable or an array as a void*, the caller should pass the total size of the item in bytes to the called function.

This is a language goal. The problem was solved years ago and is available to us straight from PL/I, here's how I envisage it looking in revised grammar: (this grammar is being actively developed currently, based on PL/I but slowly being revised as language goals solidify).

Code: [Select]

dcl matrix(10,20) string(64) varying;

func scan_for_text(table) returns bool

   arg table(*,*) string(64);

   var x = Dim(table,0);  // i.e 10
   var y = Dim(table,1);  // i.e 20

   var isup = is_uppercase(table(3,7));

end

func is_uppercase(text) returns bool

   arg text string(*);

end


Enabling dynamic access to array and varying string metadata is fundamental, I'm glad you raised it.


2. In C handling of endianess when declaring structs and bit fields required some work in order to make code portable across different architectures: Keywords and language constructs for native / big endian / little endian should be provided, and the compiler should take care of producing code for the correct endianess.

This is interesting, I know that Arm supports both types, how would you envisage this looking to a developer? what aspects of code are influenced by this?

3. In C the padding of structs requires careful design to get it right, especially when porting to different architectures. The compiler should provide simple and intuitive means for specifying and asserting the padding.

I agree, PL/I was richer in this area, taking that and revising it is a goal, controlling alignment, physical member ordering, padding, with control at the structure level and member level.

4. Design by contract (https://en.wikipedia.org/wiki/Design_by_contract) should be supported natively by the compiler so that the verification of the contracts can be enabled during compilation time and when running the unit tests. The run-time validation of the contract asserts, pre- and postcondition contracts should be controllable globally / by compilation unit / by function when creating for the actual production build. In C this can be done using macros, but this should be built-in into the language, and should be supported during run-time system.

Completely new area to me! I've read bits about it in Ada and Spark, I need to study this more, unsure of how to quantify the scope, but it could well impact grammar design so should be thought about early on.

5. Modern C compilers complain when comparing unsigned value to a signed value, which is good. But modern C compilers allow assigning unsigned variables to signed variables, and vice versa. Assigning unsigned variables to signed variables, and vice versa, should be flagged as an error by default in general. Literal values should be casted automatically by the compiler, of course. If the user wants to assign signed variables to unsigned variables, and vice versa, the operation should be done explicitly by the programmer.

Well first off, strict consistency is an absolute must, different policies in different circumstances is just asking for problems. Scenarios that can lead to unexpected, non-intuitive outcomes are a real problem, this falls into exception support area too.

6. Native 8-bit unsigned and signed data types, and a separate 8-bit char data type which is not compatible with the integer types without proper casting.

Again PL/I was attentive here, it had the 'char' data type, in essence a 'byte' with no concept of sign. For numeric use we can do this:

Code: [Select]

dcl counter bin(8) signed;
dcl totals bin(16) unsigned;
dcl rate bin(16,4) signed;
dcl interim bin(12); // defaults to signed (perhaps)


This shows the arbitrary fixed point binary types, fixed decimal is included as well.

7. Strict type checking. For example the following should not be allowed without explicit cast, because they are declared to be different data types although they are both int8_t types at the end of the day:

Code: [Select]
typedef int8_t apple_t;
typedef int8_t orange_t;

apple_t apples = 0;
orange_t oranges = 0;

apples = oranges; /* This should be flagged as an error without explicit cast */


8. More intuitive way to work with bits/bitmasks.

Type conversion is a big area here, PL/I never supported user defined type names, but it should be supported. Whether it simply be analogous to C's typedef, I'm not sure yet. It makes little sense to me though, to allow a new type name to be defined for an existing type name, no idea why C allows that.

As for bit fields and stuff, the 'bit' is a native language data type here. One can declare bit strings (akin to our familiar) character strings. And I envisage 'pad' as a type too, one can declare multiple explicit padding members if desired and all pad members can even have the same name if so desired, easing readability of code.

Thanks again for your detailed post!
Title: Re: A microcontroller programming language
Post by: Kalvin on December 19, 2022, 01:31:06 pm
Last thing for now: Checked assignments, which will be checked at run-time so that the compiler will generate code to check that the assigned value is within the allowed numerical range of the variable where the results will be assigned to. Trying to assign a value outside the allowed range will generate a run-time error. This feature should be enabled during unit testing, and should be controllable (enable/disable) globally / by compilation unit / by function during build time.

Just some random thoughts: Probably a new operator := could be defined for checked assignment. Probably a checked block could be defined so that all assignments inside the checked block will be checked during run-time if the compile-time option for the checked assignment was enabled globally or by compilation unit / by function. **. I dunno.

Edit:
** Maybe some kind of saturated assignment could be defined so that if the value to be assigned is outside the valid numerical range, the programmer can provide the upper and lower limits of a value to be assigned. If the programmer doesn't provide the limits, the compiler will use the minimum and maximum values automagically? Just thinking aloud here.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 01:54:07 pm
I know that Arm supports both types, how would you envisage this looking to a developer? what aspects of code are influenced by this?

Endianness

At early boot, SONY PSX1 (MIPS R3000), as well as POWER10, can be configured LE or BE with one bit.
Once configured, it is active and cannot be easily changed until the hardware is reset.

Sony configured the PSX1 CPU as LE. Years ago I hacked the firmware with a ROMemulator to force it as BE. Kind of toy-hack, funny to do, nothing serious.

Taylos sells their POWER10 configured as LE, so I reflashed my boss's POWER10 workstation firmware to be PPC64/BE compliant. This saved me houndred hours of work, because I only have PPC64/BE stages available and prepairing a PPC64/LE .... costs no less than two weeks.

Sure, with POWER10, MIPS5+ and some modern ARMs there are also instructions that convert the data { 16, 32 } bits LE <----> BE.

Mac_part, which is BE because designed for PowerMac, is one of the recent things I fixed on ARM64, which is LE and can't be forced BE without pain.

So, I wrote a C function to convert endianness. Easy job, you can even "inline" it. When you mind its needs, well, you find it only requires endianness conversion when mounting a partition, and it's fine that way until unmount.

Coprocessors and high-internet-working like Tulip chip can offload CPU but when they shoot ... they wants BE-only (and your target is LE) large-block data ready, so it can be worth the effort of an assembly module that directly use hardware instructions.

It depends on how often you need it.

I mean, nice to have feature, but it's not a primary need  :-//
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 02:09:24 pm
I know that Arm supports both types, how would you envisage this looking to a developer? what aspects of code are influenced by this?

Endianness

At early boot, SONY PSX1 (MIPS R3000), as well as POWER10, can be configured LE or BE with one bit.
Once configured, it is active and cannot be easily changed until the hardware is reset.

Sony configured the PSX1 CPU as LE. Years ago I hacked the firmware with a ROMemulator to force it as BE. Kind of toy-hack, funny to do, nothing serious.

Taylos sells their POWER10 configured as LE, so I reflashed my boss's POWER10 workstation firmware to be PPC64/BE compliant. This saved me houndred hours of work, because I only have PPC64/BE stages available and prepairing a PPC64/LE .... costs no less than two weeks.

Sure, with POWER10, MIPS5+ and some modern ARMs there are also instructions that convert the data { 16, 32 } bits LE <----> BE.

Mac_part, which is BE because designed for PowerMac, is one of the recent things I fixed on ARM64, which is LE and can't be forced BE without pain.

So, I wrote a C function to convert endianness. Easy job, you can even "inline" it. When you mind its needs, well, you find it only requires endianness conversion when mounting a partition, and it's fine that way until unmount.

Coprocessors and high-internet-working like Tulip chip can offload CPU but when they shoot ... they wants BE-only (and your target is LE) large-block data ready, so it can be worth the effort of an assembly module that directly use hardware instructions.

It depends on how often you need it.

I mean, nice to have feature, but it's not a primary need  :-//

So what are the code constructs that are not symmetrical with respect to endianness? For example a = b + c ; will (semantically) behave the same way yes? what code does not or would not behave symmetrically?


Title: Re: A microcontroller programming language
Post by: gnuarm on December 19, 2022, 02:14:20 pm
I don't recall, that was nearly 50 years ago.  I seem to recall one supervisor being initially upset that technically, this used a different assembler, so it must not have been a DC, rather a "hack" to the assembler code. 

No way, if it was a C file with assembly inline DC, it was an hack!

I don't think I was clear.  I'm saying the assembler tool was most likely modified to support the added instruction(s).


Quote
my-c can fix this, but it requires an advanced set of internal mechanisms to define new operators that map to new hw opcodes in a consistent way to let the compiler know their boundaries, their negative side effect (if any), provide ways to observe and verify (this is useful for ICE test cases) and also provide code rejection on error.

[] -> { ok, maybe, nothing }

But once it was thoroughly documented in the project, he was happy.

documenting hacks, well ... yes, bettern than nothing

You should never learn Forth coding.  I am confident it would blow your mind... and not in the good way.  Some people aren't bound so rigidly.

I have ZERO interest in Forth. It's not even close to what I think.
I make programs in C and Ada for a living.
Recently I am looking at RUST with interest.

Yes, I fully expected this.


Quote
Sorry, you are incomprehensible.  Can you speak English?

You said that QA doesn't complain (when) you just have to document your code.
I said that doc is always a good practice but doesn't pay money.

to pay : give someone something in return for { work done, goods received, debt incurred, favor done }

pay(someone, something) ->
{
work done: your clients(code commited, QA passed) -> return you money
favor done: your colleagues(code documented, hacks demystified) -> return you gratitude
}

I don't need you to be a dictionary.  I just need you to use the English language well.


Quote
Documenting things is always a great thing, it helps you with future work on the same code, it repays your colleagues gratitude in return (if they have to work on your code), but what only matters to you at the end of the week is whether the code passes or fails QA.

Spoken like a hack.


Quote
My customers won't pay a cent for the document, and won't pay a cent if the code fails QA.

Thus, documenting has a lower priority.
It's not good, but it's how it works.

Where you live perhaps.

One of my jobs was 100% documentation.  The only code I wrote was to support documentation. 
Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 02:27:37 pm
So what are the code constructs that are not symmetrical with respect to endianness? For example a = b + c ; will (semantically) behave the same way yes? what code does not or would not behave symmetrically?

Once BE/LE defined at boot (x86 is LE-only, it's hw wired that way), a CPU has an endianness, and so it operates untill reset.

Problems are with the load/store only when it has to access devices that has/need different endianness.
Here you need endianess conversion.

The simplest example is mac_part.
Fields are BE when you read them from the partition on the disk.
You read BE, you convert LE, so your LE-CPU (x86) can understand fields.
You keep them LE in your CPU memory space untill you need to modify something and update its copy on disk. So, you convert LE back to BE, and force sync.

With the Co-Processor, you prepare a temporary buffer, LE to BE, to feed its needs.
And you consume its block-response, BE to LE, to serve your kernel driver needs.
Title: Re: A microcontroller programming language
Post by: gnuarm on December 19, 2022, 02:29:36 pm
7. Strict type checking. For example the following should not be allowed without explicit cast, because they are declared to be different data types although they are both int8_t types at the end of the day:

Code: [Select]
typedef int8_t apple_t;
typedef int8_t orange_t;

apple_t apples = 0;
orange_t oranges = 0;

apples = oranges; /* This should be flagged as an error without explicit cast */

Such a strict type checking might be good, or not. Opinions differ.
Not even Pascal does what you are suggesting.

Would you also object to apples = apples + 1; ?
If not, why not? It's a new type (casting away the fact that typedef in C does not introduce new types, only synonyms: C11 6.7.8 §3).

Shouldn't it be apples = (apple_t)((int8_t)apples + 1);?

After all, apple_t and int8_t are no longer the same type (if they were, so would be orange_t and we are back to C semantics), so why should I expect to have the same operations defined on them.

BTW: what would be the if-then-else if-else problem?

I code with Forth  and VHDL.  Forth has types, but no type checking.  VHDL has the sorts of type checking as shown above (or close to it).  I don't know that VHDL tools have ever found an actual bug in my code through type checking.  It has caused me to do an awful lot of type conversions/casts though.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 02:31:54 pm
I don't need you to be a dictionary.  I just need you to use the English language well.
Porn website.
Spoken like a hack.
Where you live perhaps.

Alright, enjoy my ignore list.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 02:32:15 pm
Last thing for now: Checked assignments, which will be checked at run-time so that the compiler will generate code to check that the assigned value is within the allowed numerical range of the variable where the results will be assigned to. Trying to assign a value outside the allowed range will generate a run-time error. This feature should be enabled during unit testing, and should be controllable (enable/disable) globally / by compilation unit / by function during build time.

Just some random thoughts: Probably a new operator := could be defined for checked assignment. Probably a checked block could be defined so that all assignments inside the checked block will be checked during run-time if the compile-time option for the checked assignment was enabled globally or by compilation unit / by function. **. I dunno.

Edit:
** Maybe some kind of saturated assignment could be defined so that if the value to be assigned is outside the valid numerical range, the programmer can provide the upper and lower limits of a value to be assigned. If the programmer doesn't provide the limits, the compiler will use the minimum and maximum values automagically? Just thinking aloud here.

PL/I was (it seems) the first high level language to support exceptions (termed "conditions" in that language, following IBM naming conventions with their old PSW mainframe register). This was a huge help in the development of Multics and was used to confine bugs or restrict their impact. Their syntax was different to say Java or C# or C++, but that's just a detail. The way it worked was the same.

This included system generated exceptions as well as programmer defined, so I full expect that to be an inherent language feature here.

Runtime checks are important too, array bounds checks are non-negotiable, optional yes but always available if needed. There are also two distinct types of array bound checks, lets call them "shallow" and "deep".

The shallow check is very quick and checks only that the identified array element is part of the array, e.g.

Code: [Select]

dcl table(10,10) bin(16);

table(4,11) = 100; // passes shallow check because it access absolute element 51 out of 100, a valid array element.

The deep check is more costly and checks that each subscripts lies within the declared bounds. Failing a shallow check is still a bug but cannot directly corrupt memory, these are options I'd make available to the developer (or will be, if I take this forward seriously).

But I fully support these kinds of runtime checking options, enabled for new, barely exercised code and perhaps disabled in once a certain level of confidence is achieved, perhaps these options could be specified granularly too in some way as you suggest above, perhaps even enabled or disabled at runtime...


Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 02:49:50 pm
So what are the code constructs that are not symmetrical with respect to endianness? For example a = b + c ; will (semantically) behave the same way yes? what code does not or would not behave symmetrically?

Once BE/LE defined at boot (x86 is LE-only, it's hw wired that way), a CPU has an endianness, and so it operates untill reset.

Problems are with the load/store only when it has to access devices that has/need different endianness.
Here you need endianess conversion.

The simplest example is mac_part.
Fields are BE when you read them from the partition on the disk.
You read BE, you convert LE, so your LE-CPU (x86) can understand fields.
You keep them LE in your CPU memory space untill you need to modify something and update its copy on disk. So, you convert LE back to BE, and force sync.

With the Co-Processor, you prepare a temporary buffer, LE to BE, to feed its needs.
And you consume its block-response, BE to LE, to serve your kernel driver needs.

OK I see, so there's a runtime need to sometimes convert the endianness of data that's been input to the system in some way, from some external source, that's interesting. So we could allow code to discover the current endianness at runtime and also facilitate such conversions, e.g.

Code: [Select]

dcl IsBigEndian builtin;
dcl LittleEndian builtin;

// assume this data item has been populated by some IO read, and some_ptr has therefore been set

dcl partition_data bin(32) signed based(some_ptr);

if IsBigEndian() then // Both builtin functions are included for convenience, so we also have IsLittleEndian
   partition_data = LittleEndian (partition_data);
end


Is this sufficient? does this capture the gist of this? It could be extended too so that we can do this to a structure, where all endian dependent members are automatically taken care of:

Code: [Select]

data_header = LittleEndian (data_header);


Where data_header is an aggregate like an array/structure or both, this would impact numeric fields, floats and pointers/offsets even...
Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 03:26:09 pm
Yup, it seems fine  :-+
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 04:12:24 pm
For anyone interested in programming language history, I found this fascinating memoir (https://dl.acm.org/doi/pdf/10.1145/960118.808389) from one of the members of the committee who were tasked with addressing the problems of having COBOL for business and Fortran for science. IBM wanted to have a single language that could benefit both kinds of customers plus customers who were more focused on general computer science.

The costs of maintaining two distinct languages and creating compilers for their newer emerging hardware, for these two languages, was a real material cost that they wanted to address. Also (as I read in the paper) there was a growing trend where business customers were starting to do some science related coding and scientific customers were starting to do some business related, so there was definite customer desire for a single "universal" language, anyone, you can read this if it interests you...


Title: Re: A microcontroller programming language
Post by: newbrain on December 19, 2022, 04:26:10 pm
It could be extended too so that we can do this to a structure, where all endian dependent members are automatically taken care of:
Code: [Select]
data_header = LittleEndian (data_header);
Where data_header is an aggregate like an array/structure or both, this would impact numeric fields, floats and pointers/offsets even...
The version working on a single, not structured, variable is best suited to a default library, it should not be part of the language, same goes with the endianness check.

The extended version, though, looks a bit too 'magic'.

I clarify: would a user of the language be able to implement a similar fully generic function using the language itself?
It seems that some deep introspection capabilities would be needed, I don't think you mentioned them before (I might have missed them).
If not, it should be clear for the reader that this is not a regular function (BTW: I hated I could not write a write() in Pascal).

The same goes with passing arguments of different types (and this is valid also for the simple version!): the type model of this now language is totally unclear to me, as are the memory model and the execution model*.
Also: how modules are handled, though this too might have escaped my attention.

In this thread I have mostly seen syntax (not a fan, but to each their own - and not a problem unless it's obnoxious) which I think should be a lesser concern and random bits and pieces.

*Apart from the "constant cycle duration switch", which is impossible to achieve if not in the trivial cases, even forgetting about modern HW, unless someone has disproved Turing's halting theorem.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 04:32:36 pm
One final note: Classical if - then - elseif - else probelm. My suggestion is something like this:

Code: [Select]
if (expr)
  stmts;
elif (expr)
  stmts;
else
  stmts;
endif

Thus a simple test:

Code: [Select]
if (expr)
  stmts;
endif

A simple test with else clause:

Code: [Select]
if (expr)
  stmts;
else
  stmts;
endif

etc.

This is certainly an interesting grammar design question. First though when you say "problem" what do you mean? Do you mean it presents a parsing challenge or a cognitive challenge to a human reader?

The "elif" does improve readability I think, this parses fine:

Code: [Select]
if count > 100 then
   call do_something();
else
   if count < 0 then
      call do_something_else();
   else
      if count > max then
     call where_done();
              end;
           end;
end;


[attachimg=1]

But is increasingly indented, so I agree that an "elif" is very helpful here and we end up with just the one "end", let me tweak the grammar and see...
Title: Re: A microcontroller programming language
Post by: Kalvin on December 19, 2022, 05:51:06 pm
One final note: Classical if - then - elseif - else probelm. My suggestion is something like this:

Code: [Select]
if (expr)
  stmts;
elif (expr)
  stmts;
else
  stmts;
endif

Thus a simple test:

Code: [Select]
if (expr)
  stmts;
endif

A simple test with else clause:

Code: [Select]
if (expr)
  stmts;
else
  stmts;
endif

etc.

This is certainly an interesting grammar design question. First though when you say "problem" what do you mean? Do you mean it presents a parsing challenge or a cognitive challenge to a human reader?

C and C++, as well as some Algol-languages, may suffer more or less from so called dangling else syndrome.

https://en.wikipedia.org/wiki/Dangling_else

Adding this endif keyword will fix potential grammar ambiguity, help with parsing, and also help the human reader.

This is a simple error to make in C-family languages, but sometimes hard to spot:

Code: [Select]
if (expr)
  stmt;
  stmt;

When the if-statement block will always end with endif keyword, this problem will be fixed. And no braces are required, either.

Code: [Select]
if (expr)
  stmt;
  stmt;
endif

I would fix the while-statement and for-statement, as well.

Code: [Select]
while (expr)
  stmts;
endwhile

Code: [Select]
for (for-loop-control-block)
  stmts;
endfor

Whether to use endwhile and endfor, or just plain end, I do not have any strong opinion. The idea is just to avoid those easy-to-make human errors and help the parser with detecting errors.

While at talking about the C-language, grammar and braces, it would be nice if the language would be cleaned up so that the braces could be eliminated as much as possible. The function-blocks, struct and enums etc. could be defined so that no braces are required, and the end of the blocks would contain just an end keyword instead.
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 19, 2022, 06:20:07 pm
While at talking about the C-language, grammar and braces, it would be nice if the language would be cleaned up so that the braces could be eliminated as much as possible.

Why would you want to eliminate braces as much as possible? One can disagree about the exact symbol ({ } instead of something else like ( ) or begin end), but braces in syntactically C-like language do excellent job by visually and logically grouping things together - statements in code, variables in a struct, values in compound initializer, and so on. Prefer explicit over implicit; explicit grouping, in this case. Typing the braces is little work, and can be further automated in an IDE.

As a beginner, I wanted to "eliminate braces" to make code look smaller, but nowadays I find myself doing the opposite; one can group related statements in a { } block even if that isn't strictly necessary, creating a local "namespace", and making later refactoring into a function more obvious.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 07:00:14 pm
One final note: Classical if - then - elseif - else probelm. My suggestion is something like this:

Code: [Select]
if (expr)
  stmts;
elif (expr)
  stmts;
else
  stmts;
endif

Thus a simple test:

Code: [Select]
if (expr)
  stmts;
endif

A simple test with else clause:

Code: [Select]
if (expr)
  stmts;
else
  stmts;
endif

etc.

This is certainly an interesting grammar design question. First though when you say "problem" what do you mean? Do you mean it presents a parsing challenge or a cognitive challenge to a human reader?

C and C++, as well as some Algol-languages, may suffer more or less from so called dangling else syndrome.

https://en.wikipedia.org/wiki/Dangling_else

Adding this endif keyword will fix potential grammar ambiguity, help with parsing, and also help the human reader.

This is a simple error to make in C-family languages, but sometimes hard to spot:

Code: [Select]
if (expr)
  stmt;
  stmt;

When the if-statement block will always end with endif keyword, this problem will be fixed. And no braces are required, either.

Code: [Select]
if (expr)
  stmt;
  stmt;
endif

I would fix the while-statement and for-statement, as well.

Code: [Select]
while (expr)
  stmts;
endwhile

Code: [Select]
for (for-loop-control-block)
  stmts;
endfor

Whether to use endwhile and endfor, or just plain end, I do not have any strong opinion. The idea is just to avoid those easy-to-make human errors and help the parser with detecting errors.

While at talking about the C-language, grammar and braces, it would be nice if the language would be cleaned up so that the braces could be eliminated as much as possible. The function-blocks, struct and enums etc. could be defined so that no braces are required, and the end of the blocks would contain just an end keyword instead.

Well said, I've done exactly that with this new grammar, PLI used:

Code: [Select]

if expression then
   do;
   stmt;
   stmt;
   end;
else
   do;
   stmt;
   stmt;
   end;

But the "if" can be easily matched with an "end" and so removes the distinction between single statements and multiple statements within the then/else clauses.

The block delimiters just consume lines, real estate too, not a huge problem but wasteful of screen real estate.

For some time now I've argued that the C grammar has negatively impacted programming language design. It's crippled IMHO and this has had consequences for C++, Java, C# etc. adding new keywords often requires syntax acrobatics to get it right.

Check this out: https://blog.paranoidcoding.com/2022/04/11/lowercase-type-names.html
Title: Re: A microcontroller programming language
Post by: Kalvin on December 19, 2022, 07:07:41 pm
While at talking about the C-language, grammar and braces, it would be nice if the language would be cleaned up so that the braces could be eliminated as much as possible.

Why would you want to eliminate braces as much as possible? One can disagree about the exact symbol ({ } instead of something else like ( ) or begin end), but braces in syntactically C-like language do excellent job by visually and logically grouping things together - statements in code, variables in a struct, values in compound initializer, and so on. Prefer explicit over implicit; explicit grouping, in this case. Typing the braces is little work, and can be further automated in an IDE.

As a beginner, I wanted to "eliminate braces" to make code look smaller, but nowadays I find myself doing the opposite; one can group related statements in a { } block even if that isn't strictly necessary, creating a local "namespace", and making later refactoring into a function more obvious.

How many formatting styles there are for the braces, and how many fights there have been between project members how the braces should be placed in the source code.  ::) Anyway, in my opinion most of the braces are just superfluous, and they should be replaced by grammatical constructs that reduce or eliminate the use of braces as much as possible.
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 19, 2022, 07:39:07 pm
Well, not using braces sure avoids fights about how to place them in source code. :D
I still find curly braces nice block delimiters and the choice is not random, it comes from math notation.
Let people use their favorite code style, who cares. There are more important things in engineering than deciding where to put braces. It doesn't fricking matter as long as readability is preserved.

But anyway, again most of what we can read in this thread is about grammar and syntax, and very little if anything about fundamental concepts of programming.
The contract thing is one of the rare exceptions here. But it's nothing new either, it's in Ada 2012, has been in Spark for longer than this, and in Eiffel longer ago yet, and a (albeit small) number of other languages. This is definitely an interesting and important feature and definitely not trivial to implement properly.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 07:46:35 pm
There are more important things in engineering than deciding where to put braces. It doesn't fricking matter as long as readability is preserved.

Well, discuss some then.

But anyway, again most of what we can read in this thread is about grammar and syntax, and very little if anything about fundamental concepts of programming.

The contract thing is one of the rare exceptions here. But it's nothing new either, it's in Ada 2012, has been in Spark for longer than this, and in Eiffel longer ago yet, and a (albeit small) number of other languages. This is definitely an interesting and important feature and definitely not trivial to implement properly.

This is simply wrong, totally misrepresentative of the discussion, I and a few others have discussed a host of areas, like

memory alignment, field ordering, padding
big/little endian issues
coroutines and cofunctions
exception support
runtime checks
fixed point arithmetic
nested procedures/functions
argument passing
contracts
computed goto
offsets (as opposed to pointers)
array and string metadata
bit data type support

and more, all of these are mentioned somewhere or other in this thread.

Finally if you care so little about grammar, syntax, if this is so little importance to you, why not just use assembler? why would you use anything other than assembler?

Title: Re: A microcontroller programming language
Post by: DiTBho on December 19, 2022, 09:12:18 pm
Well, not using braces sure avoids fights about how to place them in source code. :D

my-c also comes with a code-beautifier.

Code: [Select]
# my-c --fo file.myc
won't compile the source, instead it will parse file.myc and output it well formatted.
(it makses sense because the code-viewer, the program that interfaces the ICE, needs the code formatted in a specific way)

problem solved ;D
Title: Re: A microcontroller programming language
Post by: brucehoult on December 19, 2022, 09:23:31 pm
I don't need you to be a dictionary.  I just need you to use the English language well.
Porn website.
Spoken like a hack.
Where you live perhaps.

Alright, enjoy my ignore list.

Indeed so. Along with Sherlock.
Title: Re: A microcontroller programming language
Post by: MIS42N on December 19, 2022, 10:48:47 pm
I'm not proposing a language, go on, check all the posts for yourself. You can't talk about language as distinct from "how it is expressed", that IS LANGUAGE. We can't even discuss languages without using a language, anyway, I'm tired, good night.
Of course language and what is being expressed are separate:
çfarë është dy plus dy
Co je dva plus dva
ni nini mbili pamoja na mbili
Kas ir divi plus divi
All express the same concept but in different languages. If we want X to be the sum of Y and Z one language will say X=Y+Z another will say push Y push Z add pop X (perhaps symbolically). What I want of a language is can it add two numbers efficiently, and I don't care what I have to say to get it to happen.

In fact the more I look at this thread the more I lean toward assembler. A recent program does a lot of arithmetic but does not need to be efficient. So I took an old PIC arithmetic package I've used for years, copied stack functionality from a previously written program, rewrote the push and pop routines in this program to use the enhanced PIC architecture but the macro remains the same. For example PushU2 fred pushes an unsigned integer onto the stack in both programs, but the underlying code is different. So it seems what I am doing is inventing a new language (or a few new words) for each program. The top octave generator had NOTEPROC x, INSERTR x, EPILOG, ONEOUT, SEVENOUT, INTEXIT. Because of the need for efficiency and timing, it was not possible to use subroutines in most places, so inline code was repeated in many places. Part of the problem being the need to keep the output aligned on 500ns boundaries. To do that required synchronizing code in the ISR to delay between 0 and 2 cycles, CALL and RETURN would require 0 to 3 cycles which was much harder to do and wasted too much time (I think the CPU utilization is around 97%).

Maybe my world view is too narrow for this thread.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 10:49:46 pm
One final note: Classical if - then - elseif - else probelm. My suggestion is something like this:

Code: [Select]
if (expr)
  stmts;
elif (expr)
  stmts;
else
  stmts;
endif

Thus a simple test:

Code: [Select]
if (expr)
  stmts;
endif

A simple test with else clause:

Code: [Select]
if (expr)
  stmts;
else
  stmts;
endif

etc.

OK I adjusted the grammar and this parses fine now:

Code: [Select]
if count > 100 then
   call do_something();
elif count < 0 then
   call do_something_else();
elif count > max then
   call where_done();
else
   return;
end;

As do any of the multitude of variants you can envisage. I like this a lot, "elif" is a very helpful addition to the grammar! Here's the parse tree for the above fragment:

[attachimg=1]

Naturally any if, else or elif clause can have one or more statements, that example has only one in each case.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 19, 2022, 11:00:34 pm
This is fascinating, this test failed to parse:

Code: [Select]
if count > 100 then
   call do_something();
elif count < 0 then
   call do_something_else();
   call do_whatever();
elif count > max then
   call where_done();
   if we_are_still_here then
      return;
   elif we_did_not_crash then
      call crash();
   else
      call stop();
else
   return;
end;

I was puzzled, staring at the source, the parse tree, the Antlr grammar etc then I noticed the embedded IF itself needed and "end" (highlighted below as uppercase)

Code: [Select]
if count > 100 then
   call do_something();
elif count < 0 then
   call do_something_else();
   call do_whatever();
elif count > max then
   call where_done();
   if we_are_still_here then
      return;
   elif we_did_not_crash then
      call crash();
   else
      call stop();
   END;
else
   return;
end;

Note to self: when testing a grammar MAKE BLOODY SURE THE INPUT TEXT IS SYNTACTICALLY VALID !

The grammar for IF is basically: IF <expression> THEN <statements>  (optional ELSE/ELIF clause) END;, i.e. every IF must have an associated END, all good, it now parses fine!

Title: Re: A microcontroller programming language
Post by: Kalvin on December 21, 2022, 07:09:22 am
About code generation and compiler optimization: There are already quite many open source compilers which provide optimized code generation, along with a full tool-chain, thus inventing a wheel may not be necessary. If the parser for a new language syntax could produce an AST which is compatible with the existing compiler technology, the customized compiler front end could be used with the existing compilers which would simplify the whole process.

The GNU compiler suite is using this approach, as there is a language-specific front-end coupled with a generic/universal back-end which is not targeted to any particular language or architecture what so ever. Unfortunately I do not know the internals of the GNU compiler suite, so I cannot provide any detailed information how hard this approach would be. However, since the source code for the GNU compiler suite is freely available, it might be possible to figure out what is takes to create a new front end for a new language. Of course any other freely available compiler suite would be a good candidate, too.

Before writing a full-blown compiler, one could create a source code translator for the new language producing valid C language (for example). This would simplify the process of trying out and prototyping different constructs in the new language.
Title: Re: A microcontroller programming language
Post by: DC1MC on December 21, 2022, 07:13:05 am
About code generation and compiler optimization: There are already quite many open source compilers which provide optimized code generation, along with a full tool-chain, thus inventing a wheel may not be necessary. If the parser for a new language syntax could produce an AST which is compatible with the existing compiler technology, the customized compiler front end could be used with the existing compilers which would simplify the whole process.

The GNU compiler suite is using this approach, as there is a language-specific front-end coupled with a generic/universal back-end which is not targeted to any particular language or architecture what so ever. Unfortunately I do not know the internals of the GNU compiler suite, so I cannot provide any detailed information how hard this approach would be. However, since the source code for the GNU compiler suite is freely available, it might be possible to figure out what is takes to create a new front end for a new language. Of course any other freely available compiler suite would be a good candidate, too.

Before writing a full-blown compiler, one could create a source code translator for the new language producing valid C language (for example). This would simplify the process of trying out and prototyping different constructs in the new language.

Yesss, yesss, translate it to C, "The only true systems languagage" ᵀᴹ , the first C++ "compilers" did the same  >:D.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 21, 2022, 09:59:38 am
About code generation and compiler optimization: There are already quite many open source compilers which provide optimized code generation, along with a full tool-chain, thus inventing a wheel may not be necessary. If the parser for a new language syntax could produce an AST which is compatible with the existing compiler technology, the customized compiler front end could be used with the existing compilers which would simplify the whole process.

The GNU compiler suite is using this approach, as there is a language-specific front-end coupled with a generic/universal back-end which is not targeted to any particular language or architecture what so ever. Unfortunately I do not know the internals of the GNU compiler suite, so I cannot provide any detailed information how hard this approach would be. However, since the source code for the GNU compiler suite is freely available, it might be possible to figure out what is takes to create a new front end for a new language. Of course any other freely available compiler suite would be a good candidate, too.

Before writing a full-blown compiler, one could create a source code translator for the new language producing valid C language (for example). This would simplify the process of trying out and prototyping different constructs in the new language.

Yesss, yesss, translate it to C, "The only true systems languagage" ᵀᴹ , the first C++ "compilers" did the same  >:D.

So did Kyoto Common Lisp.

Nowadays people tend to use a well-defined architecture-neutral intermediate representation (https://en.wikipedia.org/wiki/List_of_Java_bytecode_instructions) that is available, supported (and debugged) on a variety of architectures. It even has multithreading and multiprocessing.
Title: Re: A microcontroller programming language
Post by: brucehoult on December 21, 2022, 11:52:49 am
About code generation and compiler optimization: There are already quite many open source compilers which provide optimized code generation, along with a full tool-chain, thus inventing a wheel may not be necessary. If the parser for a new language syntax could produce an AST which is compatible with the existing compiler technology, the customized compiler front end could be used with the existing compilers which would simplify the whole process.

The GNU compiler suite is using this approach, as there is a language-specific front-end coupled with a generic/universal back-end which is not targeted to any particular language or architecture what so ever. Unfortunately I do not know the internals of the GNU compiler suite, so I cannot provide any detailed information how hard this approach would be. However, since the source code for the GNU compiler suite is freely available, it might be possible to figure out what is takes to create a new front end for a new language. Of course any other freely available compiler suite would be a good candidate, too.

Before writing a full-blown compiler, one could create a source code translator for the new language producing valid C language (for example). This would simplify the process of trying out and prototyping different constructs in the new language.

Yesss, yesss, translate it to C, "The only true systems languagage" ᵀᴹ , the first C++ "compilers" did the same  >:D.

There is no reason to put the word "compilers" in scare quotes. Compiling to C is every bit as valid as compiling to assembly language.

The question to ask is this: if you make an error in your C++ code, will you get an error message from the C++-to-C compiler, or from the C compiler?

If the answer is ever "the C compiler" then that's a bug in the C++-to-C compiler, plain and simple.

And it didn't happen with CFront. Not even in 1989.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 21, 2022, 12:39:11 pm
So far as code generators go, LLVM seems to be the ideal way to do that. I've never used it but it seems hugely capable, so were I taking this forward in earnest, that would be my focus. Interestingly too Antlr4 does not support automatic AST building, so some effort seems likely to deal with that (it generates a grammar derivation tree "parse tree").

Generating C is an attractive idea too, it could serve as a proof of concept. But this would still be some serious work, it would require a C abstract machine idea of sorts, that would need to expose stack, push pop to/from the stack, etc. Unless one wanted to simply translate code fragments into C, but that assumes a great deal, the semantics of C can't be assumed to be the semantics of the source language.

For example C has no function nesting, representing that - in a general way - in C requires abstracting the concept, this is why a more machine like abstraction might be better, in fact generating assembly language embedded in C might be better then a C dev environment could be used as is.

One could craft a set of C functions that generate standard chunks of assembler for block entry, exit and so on. It might even be possible to build a contiguous block of memory this way and then forcibly begin execution of it once done. Rather than generating inline assembler, generate it into some address, some buffer...in fact that sounds a lot like a useful new language feature!
Title: Re: A microcontroller programming language
Post by: Kalvin on December 21, 2022, 12:49:35 pm
For example C has no function nesting, representing that - in a general way - in C requires abstracting the concept, this is why a more machine like abstraction might be better, in fact generating assembly language embedded in C might be better then a C dev environment could be used as is.

GNU C has an extension for function nesting.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 21, 2022, 01:03:47 pm
https://asmjit.com/
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 21, 2022, 02:17:59 pm
I've managed to get Antlr to recognize the grammar's support for numeric literals in multiple bases and which can contain readability spaces (or underscore) as separators. All of these are recognized:

Code: [Select]
proc test(x)

value = DEF ABC 2DC0 6BA0:h * 100; // note, no more than a single space can be the separator.

call reboot(FACE DEFB 10C0:H + 239:H);

value = 100 123.45;

value = 1011 FC4.7F:h;

value = 1010 1010;

value = 123:d;

value = 1010 1010:b;

octal = 765:o;

end;


There are four kinds of numeric literals, binary, octal, decimal and hex, they all need a colon-char base designator except dec, for which this can be omitted.

Title: Re: A microcontroller programming language
Post by: Kalvin on December 22, 2022, 09:14:52 am
I've managed to get Antlr to recognize the grammar's support for numeric literals in multiple bases and which can contain readability spaces (or underscore) as separators. All of these are recognized:

Code: [Select]
proc test(x)

value = DEF ABC 2DC0 6BA0:h * 100; // note, no more than a single space can be the separator.

call reboot(FACE DEFB 10C0:H + 239:H);

value = 100 123.45;

value = 1011 FC4.7F:h;

value = 1010 1010;

value = 123:d;

value = 1010 1010:b;

octal = 765:o;

end;


There are four kinds of numeric literals, binary, octal, decimal and hex, they all need a colon-char base designator except dec, for which this can be omitted.

Personal opinion: I do not like spaces in the literals. Underscore, for example, would be better.

From the tokenizer/parser point of view things would be much easier if the spaces would not be allowed in literals. See also my comment about the tools below.

Edit: There has also been some discussion on how to group the digits in numerical literals. I guess that it would be possible to build a simple tool using regexpr rule(s) for checking the grouping of the digits for floating point, decimal, hex, octal and binary numbers of person's liking. Just run the checker as part of the build process, and the checker will fail the build process if the numerical literals are using invalid grouping. Building a directive for the rules of digit grouping into the language grammar could be also possible, but I am not sure if that would be a wise choice (without extensive prototyping and evaluation at least).

---

At some point in this thread there was a discussion whether to use reserved words or not. I think the history of the programming languages have shown that using reserved words makes the grammar easier to parse and reduce ambiguity. Just keep the grammar as simple as possible.

About tools: All tools - such as code formatters, IDEs, refactoring tools etc. - would benefit from a simple grammar without ambiguity.

We have something like 70 years of knowledge about programming languages, so there is no need to repeat the problems of the earlier programming languages.
Title: Re: A microcontroller programming language
Post by: brucehoult on December 22, 2022, 09:37:54 am
For example C has no function nesting, representing that - in a general way - in C requires abstracting the concept, this is why a more machine like abstraction might be better, in fact generating assembly language embedded in C might be better then a C dev environment could be used as is.

GNU C has an extension for function nesting.

You don't need to generate nested functions in the C output to implement source language nested functions.
Title: Re: A microcontroller programming language
Post by: Kalvin on December 22, 2022, 09:48:37 am
For example C has no function nesting, representing that - in a general way - in C requires abstracting the concept, this is why a more machine like abstraction might be better, in fact generating assembly language embedded in C might be better then a C dev environment could be used as is.

GNU C has an extension for function nesting.

You don't need to generate nested functions in the C output to implement source language nested functions.

But it would be much easier if the compiler uses the GNU C extension for the nested functions. The nested functions should be able to access the context of the surrounding function(s). Without nesting, accessing the context of the surrounding function(s) may get really difficult to implement.
Title: Re: A microcontroller programming language
Post by: brucehoult on December 22, 2022, 10:56:00 am
For example C has no function nesting, representing that - in a general way - in C requires abstracting the concept, this is why a more machine like abstraction might be better, in fact generating assembly language embedded in C might be better then a C dev environment could be used as is.

GNU C has an extension for function nesting.

You don't need to generate nested functions in the C output to implement source language nested functions.

But it would be much easier if the compiler uses the GNU C extension for the nested functions. The nested functions should be able to access the context of the surrounding function(s). Without nesting, accessing the context of the surrounding function(s) may get really difficult to implement.

Easier, but maybe you don't like the way that compiler implements it, or you want to be able to use other C compilers too (MSVC, SDCC, whatever...)

And it's not hard at all. For example each function declares a struct type with all its local variables in, then declares just one actual local variable of that type. The address of it can be passed as an extra argument to "nested" functions ("static link"), or stored in a global array ("display").

Or only put the variables that are actually used by the nested function in that struct. Or allocate that struct on the heap not the stack so you have a proper closure, not simply nested functions -- that's a common language feature that you CAN'T implement using gcc nested functions.

This is all completely trivial compared to the rest of the work of writing a compiler.
Title: Re: A microcontroller programming language
Post by: Kalvin on December 22, 2022, 12:11:10 pm
For example C has no function nesting, representing that - in a general way - in C requires abstracting the concept, this is why a more machine like abstraction might be better, in fact generating assembly language embedded in C might be better then a C dev environment could be used as is.

GNU C has an extension for function nesting.

You don't need to generate nested functions in the C output to implement source language nested functions.

But it would be much easier if the compiler uses the GNU C extension for the nested functions. The nested functions should be able to access the context of the surrounding function(s). Without nesting, accessing the context of the surrounding function(s) may get really difficult to implement.

Easier, but maybe you don't like the way that compiler implements it, or you want to be able to use other C compilers too (MSVC, SDCC, whatever...)

And it's not hard at all. For example each function declares a struct type with all its local variables in, then declares just one actual local variable of that type. The address of it can be passed as an extra argument to "nested" functions ("static link"), or stored in a global array ("display").

Or only put the variables that are actually used by the nested function in that struct. Or allocate that struct on the heap not the stack so you have a proper closure, not simply nested functions -- that's a common language feature that you CAN'T implement using gcc nested functions.

This is all completely trivial compared to the rest of the work of writing a compiler.

Although the nested functions are handy and helpful, it is possible to create programs without them. Therefore the nested functions should probably be included in the grammar only as new language feature in the future.
Title: Re: A microcontroller programming language
Post by: Kalvin on December 22, 2022, 12:47:03 pm
<snip>
memory alignment, field ordering, padding
big/little endian issues
coroutines and cofunctions
exception support
runtime checks
fixed point arithmetic
nested procedures/functions
argument passing
contracts
computed goto
offsets (as opposed to pointers)
array and string metadata
bit data type support
<snip>

in the spirit of C++, any plans for passing an argument of a function as reference, too? That would remove the need of using the dereference-operator when accessing a variable inside the function that was passed as a reference. Passing arguments as a reference will also remove the need for checking NULL pointer values for the arguments.

Talking about the NULL pointers: Please introduce a proper keyword for denoting a NULL pointer value, for example null or nullptr.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 22, 2022, 12:58:09 pm
I've managed to get Antlr to recognize the grammar's support for numeric literals in multiple bases and which can contain readability spaces (or underscore) as separators. All of these are recognized:

Code: [Select]
proc test(x)

value = DEF ABC 2DC0 6BA0:h * 100; // note, no more than a single space can be the separator.

call reboot(FACE DEFB 10C0:H + 239:H);

value = 100 123.45;

value = 1011 FC4.7F:h;

value = 1010 1010;

value = 123:d;

value = 1010 1010:b;

octal = 765:o;

end;


There are four kinds of numeric literals, binary, octal, decimal and hex, they all need a colon-char base designator except dec, for which this can be omitted.

Personal opinion: I do not like spaces in the literals. Underscore, for example, would be better.

From the tokenizer/parser point of view things would be much easier if the spaces would not be allowed in literals. See also my comment about the tools below.

Edit: There has also been some discussion on how to group the digits in numerical literals. I guess that it would be possible to build a simple tool using regexpr rule(s) for checking the grouping of the digits for floating point, decimal, hex, octal and binary numbers of person's liking. Just run the checker as part of the build process, and the checker will fail the build process if the numerical literals are using invalid grouping. Building a directive for the rules of digit grouping into the language grammar could be also possible, but I am not sure if that would be a wise choice (without extensive prototyping and evaluation at least).

---

At some point in this thread there was a discussion whether to use reserved words or not. I think the history of the programming languages have shown that using reserved words makes the grammar easier to parse and reduce ambiguity. Just keep the grammar as simple as possible.

About tools: All tools - such as code formatters, IDEs, refactoring tools etc. - would benefit from a simple grammar without ambiguity.

We have something like 70 years of knowledge about programming languages, so there is no need to repeat the problems of the earlier programming languages.

The (single) space is entirely optional and underscore also permitted. There are standards bodies who recommend spaces over commas or periods, the space is actually common in several industries and reduces risk of misunderstanding across cultures, I did research this, its an interesting aspect of language design.

Having the type as suffix (e.g :H) rather than a prefix (e.g. 0x) is what makes these spaces possible, more effort to recognize but Antlr4 is extremely powerful.

The reserved words question gets much attention, the core motive is never ever fail to compile code that might have identifiers that are the same as newly added language keywords. Consider C# where they had to use "yield return" when simply "yield" was the obvious choice, their grammar could not support this and be backward compatible.

That's frankly a poor design right there, but like most languages the designers pay too little attention to careful methodical grammar design, grammar's must be designed, no escaping this point, the trend for years has been "make it C-like" and just look at the problems that's led to.

Of course avoiding, minimizing obfuscation is important too, and providing compile time options to warn against unintended keyword clash is easy to do and better then losing backward compatibility.

I want to stress that the grammar has no ambiguities, C++ even C do have grammar ambiguities, you can read about these, C++ also has the most complex grammar of any programming language, littered with edge cases and ifs and buts, until recently this construct was illegal in C++

Code: [Select]

Typename<Type<Other>>


Instead one was forced to write

Code: [Select]

Typename<Type<Other> >


That's frankly ridiculous (the >> is recognized as a shift operator) and can be traced back to sloppy or hasty grammar choices, I talk about this a lot but that's because its very important, without careful design you get all the syntax mess we see in C++, C# and so on.

This new grammar parses robustly, easily that's because there are only two fundamental kinds of statements, assignments and keyword statements.

An assignment (however complex lexically) can always be recognized 100% reliably irrespective of the spelling of terms. Only if a statement is not an assignment do we look for a keyword start.

Despite first impressions, this is actually simple, not complex, grammars should have this kind of power IMHO.









Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 22, 2022, 01:06:54 pm
<snip>
memory alignment, field ordering, padding
big/little endian issues
coroutines and cofunctions
exception support
runtime checks
fixed point arithmetic
nested procedures/functions
argument passing
contracts
computed goto
offsets (as opposed to pointers)
array and string metadata
bit data type support
<snip>

in the spirit of C++, any plans for passing an argument of a function as reference, too? That would remove the need of using the dereference-operator when accessing a variable inside the function that was passed as a reference. Passing arguments as a reference will also remove the need for checking NULL pointer values for the arguments.

Talking about the NULL pointers: Please introduce a proper keyword for denoting a NULL pointer value, for example null or nullptr.

Yes, all args are passed by ref. To achieve pass by value (that is, pass a ref to a temp copy) one just surrounds the arg in parens:

Code: [Select]

call set_timer((timer_id));


That makes the arg into an expression and expressions are always evaluated into a temporary and a ref to that temp then passed. No need for & and * all over the place!
Title: Re: A microcontroller programming language
Post by: Kalvin on December 22, 2022, 01:40:32 pm
<snip>
memory alignment, field ordering, padding
big/little endian issues
coroutines and cofunctions
exception support
runtime checks
fixed point arithmetic
nested procedures/functions
argument passing
contracts
computed goto
offsets (as opposed to pointers)
array and string metadata
bit data type support
<snip>

in the spirit of C++, any plans for passing an argument of a function as reference, too? That would remove the need of using the dereference-operator when accessing a variable inside the function that was passed as a reference. Passing arguments as a reference will also remove the need for checking NULL pointer values for the arguments.

Talking about the NULL pointers: Please introduce a proper keyword for denoting a NULL pointer value, for example null or nullptr.

Yes, all args are passed by ref. To achieve pass by value (that is, pass a ref to a temp copy) one just surrounds the arg in parens:

Code: [Select]

call set_timer((timer_id));


That makes the arg into an expression and expressions are always evaluated into a temporary and a ref to that temp then passed. No need for & and * all over the place!

The function declaration should be the place where it is defined whether an argument is passed by value or by reference.
Title: Re: A microcontroller programming language
Post by: Kalvin on December 22, 2022, 02:14:31 pm
I want to stress that the grammar has no ambiguities, C++ even C do have grammar ambiguities, you can read about these, C++ also has the most complex grammar of any programming language, littered with edge cases and ifs and buts, until recently this construct was illegal in C++

Code: [Select]
Typename<Type<Other>>

Instead one was forced to write

Code: [Select]
Typename<Type<Other> >

That's frankly ridiculous (the >> is recognized as a shift operator) and can be traced back to sloppy or hasty grammar choices, I talk about this a lot but that's because its very important, without careful design you get all the syntax mess we see in C++, C# and so on.

Ada has a major blooper too as they are using parenthesis () for array index instead of brackets []. Now it is impossible to determine whether a(n) is a function call or an array element access, without checking the actual definition.

Crafting a programming language is really hard, and it is very easy to make mistakes that may carry on for decades.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 22, 2022, 02:26:51 pm
<snip>
memory alignment, field ordering, padding
big/little endian issues
coroutines and cofunctions
exception support
runtime checks
fixed point arithmetic
nested procedures/functions
argument passing
contracts
computed goto
offsets (as opposed to pointers)
array and string metadata
bit data type support
<snip>

in the spirit of C++, any plans for passing an argument of a function as reference, too? That would remove the need of using the dereference-operator when accessing a variable inside the function that was passed as a reference. Passing arguments as a reference will also remove the need for checking NULL pointer values for the arguments.

Talking about the NULL pointers: Please introduce a proper keyword for denoting a NULL pointer value, for example null or nullptr.

Yes, all args are passed by ref. To achieve pass by value (that is, pass a ref to a temp copy) one just surrounds the arg in parens:

Code: [Select]

call set_timer((timer_id));


That makes the arg into an expression and expressions are always evaluated into a temporary and a ref to that temp then passed. No need for & and * all over the place!

The function declaration should be the place where it is defined whether an argument is passed by value or by reference.

That's a possibility I agree. But then we must sacrifice the ability to ever pass by ref to some code, just because we might have some scenarios where we don't want the caller to modify the arg.


Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 22, 2022, 02:32:03 pm
I want to stress that the grammar has no ambiguities, C++ even C do have grammar ambiguities, you can read about these, C++ also has the most complex grammar of any programming language, littered with edge cases and ifs and buts, until recently this construct was illegal in C++

Code: [Select]
Typename<Type<Other>>

Instead one was forced to write

Code: [Select]
Typename<Type<Other> >

That's frankly ridiculous (the >> is recognized as a shift operator) and can be traced back to sloppy or hasty grammar choices, I talk about this a lot but that's because its very important, without careful design you get all the syntax mess we see in C++, C# and so on.

Ada has a major blooper too as they are using parenthesis () for array index instead of brackets []. Now it is impossible to determine whether a(n) is a function call or an array element access, without checking the actual definition.

Crafting a programming language is really hard, and it is very easy to make mistakes that may carry on for decades.

Well that's an interesting point, PL/I (and my draft grammar) do the same as Ada, I've never thought much about this, like how important is it to be able to differentiate, modern IDE technology could easily render arrays in a different color to functions too. I'll think about this...
Title: Re: A microcontroller programming language
Post by: tggzzz on December 22, 2022, 02:48:29 pm
Talking about the NULL pointers: Please introduce a proper keyword for denoting a NULL pointer value, for example null or nullptr.

Or avoid "the billion dollar mistake" altogether. https://en.wikipedia.org/wiki/Null_pointer#History

That would be far more useful than diddling with syntax.
Title: Re: A microcontroller programming language
Post by: Kalvin on December 22, 2022, 02:52:29 pm
Talking about the NULL pointers: Please introduce a proper keyword for denoting a NULL pointer value, for example null or nullptr.

Or avoid "the billion dollar mistake" altogether.
https://en.wikipedia.org/wiki/Null_pointer#History

I agree 100%.

Edit: Probably Ada or some other safety-driven language might provide a guidance how to implement this well in the language grammar.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 22, 2022, 03:03:37 pm
Why not provide a construct that directly helps people doing low-level embedded programming?

For example, a select statement that is similar to a C switch statement:
Code: [Select]
select {
    case event1
        // handle the event
        ...
        break ;
    case event2
        // handle the event
        ...
        break ;
}
That statement would pause until either of the events occur and then execute the code within the relevant case. Although the select waits on several events, only one of the events would be handled by the statement when an event occurs.

An event could be any of:

As anyone who has done embedded programming will recognise, with those constructs a vast range of embedded algorithms can be simply, easily, and understandably implemented.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 22, 2022, 04:44:36 pm
Why not provide a construct that directly helps people doing low-level embedded programming?

For example, a select statement that is similar to a C switch statement:
Code: [Select]
select {
    case event1
        // handle the event
        ...
        break ;
    case event2
        // handle the event
        ...
        break ;
}
That statement would pause until either of the events occur and then execute the code within the relevant case. Although the select waits on several events, only one of the events would be handled by the statement when an event occurs.

An event could be any of:
  • input available
  • output completed
  • a timeout
  • a message from another thread

As anyone who has done embedded programming will recognise, with those constructs a vast range of embedded algorithms can be simply, easily, and understandably implemented.

Once you start to talk about events, time, resume, suspend etc then we're no longer talking language but OS.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 22, 2022, 04:58:44 pm
Talking about the NULL pointers: Please introduce a proper keyword for denoting a NULL pointer value, for example null or nullptr.

Or avoid "the billion dollar mistake" altogether. https://en.wikipedia.org/wiki/Null_pointer#History

That would be far more useful than diddling with syntax.

Yes I spent a little bit of time looking at that. PL/I was an early language that supported pointers and they are opaque. You cannot do pointer arithmetic in PL/I (or Ada) nor would I want to support that either. This is the core of many of C's problems the ease with which the programmer can create addresses.

I think 99% of the problems would vanish when pointer arithmetic is removed. So I must ask, what do people need this for when using C? Why is it so relevant? What downsides would ensue if this were not possible? what would need to be included in a language to compensate for loss of pointer arithmetic and the ability to set pointers to hard numeric values?
Title: Re: A microcontroller programming language
Post by: brucehoult on December 22, 2022, 05:18:46 pm
Yes, all args are passed by ref. To achieve pass by value (that is, pass a ref to a temp copy) one just surrounds the arg in parens:

omfg. I am without words.

You just threw away a huge chunk of performance, as well as compatibility with all existing libraries and OS interfaces.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 22, 2022, 05:59:33 pm
Why not provide a construct that directly helps people doing low-level embedded programming?

For example, a select statement that is similar to a C switch statement:
Code: [Select]
select {
    case event1
        // handle the event
        ...
        break ;
    case event2
        // handle the event
        ...
        break ;
}
That statement would pause until either of the events occur and then execute the code within the relevant case. Although the select waits on several events, only one of the events would be handled by the statement when an event occurs.

An event could be any of:
  • input available
  • output completed
  • a timeout
  • a message from another thread

As anyone who has done embedded programming will recognise, with those constructs a vast range of embedded algorithms can be simply, easily, and understandably implemented.

Once you start to talk about events, time, resume, suspend etc then we're no longer talking language but OS.

Sorry, that's simply not true for multiple languages, Ada and SPARK amongst others.

Some processors have that functionality implemented in hardware. No RTOS necessary. And very good they are too for real-time embedded applications.

I you started this thread because you are interested in language features that would help embedded applications. Has that changed?

I've been pondering the idea of attempting to create a new (compiled) programming language specifically designed for hardware programming, microcontrollers
....
So I could start by asking what people's views are here, what do you - as an MCU developer - find good...
Title: Re: A microcontroller programming language
Post by: Siwastaja on December 22, 2022, 06:04:30 pm
Ada has a major blooper too as they are using parenthesis () for array index instead of brackets []. Now it is impossible to determine whether a(n) is a function call or an array element access, without checking the actual definition.

Yeah, I hate this kind of implicit "you have to know what X is, or have an IDE tell you" thing. I don't do C++ a lot, but when I do, not once I have mistaken function argument that is passed as reference vs. passed by value. There is difference (added &) in function definition, but at call site it looks exactly the same! Yet I think this is such large fundamental difference (whether my local variable might get modified or not) that I, as a caller, want to make that distinction, so I like the C pointer way of having to do that. (Minus points for C for the "arrays decay as pointers automagically" feature, kinda ruins it.)
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 22, 2022, 06:38:43 pm
Yes, all args are passed by ref. To achieve pass by value (that is, pass a ref to a temp copy) one just surrounds the arg in parens:

omfg. I am without words.

You just threw away a huge chunk of performance, as well as compatibility with all existing libraries and OS interfaces.

Ha! but not without expletives!

OK since you do want to discuss this with me, let me respond.

I'm referring to the general policy for arg passing, ultimately an implementation could do this in various ways (pass in a register for example). I'm not really referring to the implementation but to the semantics, I don't think I made that clear.

So "pass by reference" here is meant to indicate that the callee can change something in the datum that's been passed. That's the default convention, and if the the caller want's to prevent a callee from changing the value of an arg, they'd simply make that arg an expression with ( and ).

This leads to increased performance because unless the developer wants to pass by value, they can write the code and just let args pass by ref, meaning passing a bunch of structures, strings, arrays or combinations of these will entail nothing more than pushing a pointer onto the stack!

Look at how often C code passes args by address too, I mean most code wants to pass by address so why not make that the default mode and reserve the extra syntax for the less common cases where we want to pass by value? Instead we have calling code littered with & and called code littered with *.

As for compatibility with other languages I've said nothing about interop, naturally when calling code written in a different language then that code's convention must be honored.


Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 22, 2022, 06:57:58 pm
Ada has a major blooper too as they are using parenthesis () for array index instead of brackets []. Now it is impossible to determine whether a(n) is a function call or an array element access, without checking the actual definition.

Yeah, I hate this kind of implicit "you have to know what X is, or have an IDE tell you" thing. I don't do C++ a lot, but when I do, not once I have mistaken function argument that is passed as reference vs. passed by value. There is difference (added &) in function definition, but at call site it looks exactly the same! Yet I think this is such large fundamental difference (whether my local variable might get modified or not) that I, as a caller, want to make that distinction, so I like the C pointer way of having to do that. (Minus points for C for the "arrays decay as pointers automagically" feature, kinda ruins it.)

This is far from clear cut. After all an array and a function are in an abstract sense the same, they are (mathematically) mappings from an input space to an output space. One might also remove an array and implement it as a function (or vice verse)  in some code update, all the code would remain unchanged, no need to replace [ and ] with ( and ) all over the place.

Why does one need to care about how such a mapping is implemented when looking at code? we're putting something in and getting something out, how that's done should (IMHO) have no bearing on the calling code's comprehension.

Functional programming actually leverages a concept of "memoization", where a call to a function with a set of args, has the result cached so that a later call with the same args, returns the same result minus the computational overhead of having to evaluate it again, an array is a great way to implement memoization in many cases.

So I'm just not convinced myself, its an easy thing to do, altering the grammar for that is almost trivial, but you then run the risk of losing symbolic characters that might have some better use in the future, there really aren't that many standard (ASCII) symbols available.

Title: Re: A microcontroller programming language
Post by: Kalvin on December 22, 2022, 07:28:21 pm
Ada has a major blooper too as they are using parenthesis () for array index instead of brackets []. Now it is impossible to determine whether a(n) is a function call or an array element access, without checking the actual definition.

Yeah, I hate this kind of implicit "you have to know what X is, or have an IDE tell you" thing. I don't do C++ a lot, but when I do, not once I have mistaken function argument that is passed as reference vs. passed by value. There is difference (added &) in function definition, but at call site it looks exactly the same! Yet I think this is such large fundamental difference (whether my local variable might get modified or not) that I, as a caller, want to make that distinction, so I like the C pointer way of having to do that. (Minus points for C for the "arrays decay as pointers automagically" feature, kinda ruins it.)

This is far from clear cut. After all an array and a function are in an abstract sense the same, they are (mathematically) mappings from an input space to an output space. One might also remove an array and implement it as a function (or vice verse)  in some code update, all the code would remain unchanged, no need to replace [ and ] with ( and ) all over the place.

Why does one need to care about how such a mapping is implemented when looking at code? we're putting something in and getting something out, how that's done should (IMHO) have no bearing on the code comprehension.

So I'm just not convinced myself, its an easy thing to do, altering the grammar for that is almost trivial, but you then run the risk of losing symbolic characters that might have some better use in the future, there really aren't that many standard (ASCII) symbols available.

If you define brackets [] as an operator, that would keep some possibilities open for future language extensions. For example in C++ it is possible to define operator [] for a new class.

You can argue that a function call and array indexing are both mappings, but in practice (ie. the code generated by the compiler) accessing an array element is fundamentally different from a function call. In summary: using different symbols for indexing and function arguments makes good sense, and there is no reason to repeat the Ada blunder again.

This might be interesting reading: Ada 83 to Ada 2012: lessons learned over 30 years of language design
https://www.researchgate.net/profile/S-Taft/publication/286243707_Ada_83_to_Ada_2012/links/56e02f1408aec4b3333d0201/Ada-83-to-Ada-2012.pdf (https://www.researchgate.net/profile/S-Taft/publication/286243707_Ada_83_to_Ada_2012/links/56e02f1408aec4b3333d0201/Ada-83-to-Ada-2012.pdf) Unfortunately I do not have access to the text of the actual presentation.

This may be a little controversial: It is now year 2022, and modern OSs have support for Unicode character set. Basically there is no reason to use plain ASCII character set, so the language could use some symbols defined in Unicode character set. To be clear here, I do not want to see a new APL either. :)
Title: Re: A microcontroller programming language
Post by: tggzzz on December 22, 2022, 07:46:34 pm
Why not provide a construct that directly helps people doing low-level embedded programming?

For example, a select statement that is similar to a C switch statement:
Code: [Select]
select {
    case event1
        // handle the event
        ...
        break ;
    case event2
        // handle the event
        ...
        break ;
}
That statement would pause until either of the events occur and then execute the code within the relevant case. Although the select waits on several events, only one of the events would be handled by the statement when an event occurs.

An event could be any of:
  • input available
  • output completed
  • a timeout
  • a message from another thread

As anyone who has done embedded programming will recognise, with those constructs a vast range of embedded algorithms can be simply, easily, and understandably implemented.

Once you start to talk about events, time, resume, suspend etc then we're no longer talking language but OS.

Sorry, that's simply not true for multiple languages, Ada and SPARK amongst others.

Some processors have that functionality implemented in hardware. No RTOS necessary. And very good they are too for real-time embedded applications.

I you started this thread because you are interested in language features that would help embedded applications. Has that changed?

No nothing has changed, and there might be some merit in what you're suggesting, I might be wrong. My position is though that to start designing such capabilities into the language begins to impose a particular design pattern for events, threads, scheduling and so on and that might be very restrictive to the broader community of users.

But you are already doing exactly that with the features you are proposing including/excluding from your language. Other languages also, necessarily, do that in their own ways too (Examples: statistics processing languages, modelling languages, HDLs, business-oriented languages, even spreadsheets :) )

Quote
Better I feel for the designer to be free to select such an important aspect of a platform and just rely on the language to help them interact with that platform, but I might be wrong, I'd need to look into this much more and I guess that's a large scope creep at this point (but rest assured we could add keywords for some of the stuff without any problems).

It is indeed scope creep. But then every departure from keywords and syntax into useful behaviour is also scope creep.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 22, 2022, 08:31:38 pm
Why not provide a construct that directly helps people doing low-level embedded programming?

For example, a select statement that is similar to a C switch statement:
Code: [Select]
select {
    case event1
        // handle the event
        ...
        break ;
    case event2
        // handle the event
        ...
        break ;
}
That statement would pause until either of the events occur and then execute the code within the relevant case. Although the select waits on several events, only one of the events would be handled by the statement when an event occurs.

An event could be any of:
  • input available
  • output completed
  • a timeout
  • a message from another thread

As anyone who has done embedded programming will recognise, with those constructs a vast range of embedded algorithms can be simply, easily, and understandably implemented.

Once you start to talk about events, time, resume, suspend etc then we're no longer talking language but OS.

Sorry, that's simply not true for multiple languages, Ada and SPARK amongst others.

Some processors have that functionality implemented in hardware. No RTOS necessary. And very good they are too for real-time embedded applications.

I you started this thread because you are interested in language features that would help embedded applications. Has that changed?

No nothing has changed, and there might be some merit in what you're suggesting, I might be wrong. My position is though that to start designing such capabilities into the language begins to impose a particular design pattern for events, threads, scheduling and so on and that might be very restrictive to the broader community of users.

But you are already doing exactly that with the features you are proposing including/excluding from your language. Other languages also, necessarily, do that in their own ways too (Examples: statistics processing languages, modelling languages, HDLs, business-oriented languages, even spreadsheets :) )

Quote
Better I feel for the designer to be free to select such an important aspect of a platform and just rely on the language to help them interact with that platform, but I might be wrong, I'd need to look into this much more and I guess that's a large scope creep at this point (but rest assured we could add keywords for some of the stuff without any problems).

It is indeed scope creep. But then every departure from keywords and syntax into useful behaviour is also scope creep.

Thanks.

Languages describe behavior, so before you can describe behavior, represent it, you need language and languages have - need - grammar - so I don't quite understand dismissive remarks about the significance of grammar. Being dismissive, glib about this aspect of programming languages is why we have so many problems with C++ and various other languages. If the grammar is crippled you pay a long term price and that can be a high price as the many examples I've shared show.

I mean it seems like I'm hearing this sometimes "I want to talk about programming languages but do not care about the superficial grammar, syntax BS" - to which I can only reply "Very well, in that case use assembler".

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on December 22, 2022, 08:39:22 pm
Ada has a major blooper too as they are using parenthesis () for array index instead of brackets []. Now it is impossible to determine whether a(n) is a function call or an array element access, without checking the actual definition.

Yeah, I hate this kind of implicit "you have to know what X is, or have an IDE tell you" thing. I don't do C++ a lot, but when I do, not once I have mistaken function argument that is passed as reference vs. passed by value. There is difference (added &) in function definition, but at call site it looks exactly the same! Yet I think this is such large fundamental difference (whether my local variable might get modified or not) that I, as a caller, want to make that distinction, so I like the C pointer way of having to do that. (Minus points for C for the "arrays decay as pointers automagically" feature, kinda ruins it.)

This is far from clear cut. After all an array and a function are in an abstract sense the same, they are (mathematically) mappings from an input space to an output space. One might also remove an array and implement it as a function (or vice verse)  in some code update, all the code would remain unchanged, no need to replace [ and ] with ( and ) all over the place.

Why does one need to care about how such a mapping is implemented when looking at code? we're putting something in and getting something out, how that's done should (IMHO) have no bearing on the code comprehension.

So I'm just not convinced myself, its an easy thing to do, altering the grammar for that is almost trivial, but you then run the risk of losing symbolic characters that might have some better use in the future, there really aren't that many standard (ASCII) symbols available.

If you define brackets [] as an operator, that would keep some possibilities open for future language extensions. For example in C++ it is possible to define operator [] for a new class.

You can argue that a function call and array indexing are both mappings, but in practice (ie. the code generated by the compiler) accessing an array element is fundamentally different from a function call. In summary: using different symbols for indexing and function arguments makes good sense, and there is no reason to repeat the Ada blunder again.

This might be interesting reading: Ada 83 to Ada 2012: lessons learned over 30 years of language design
https://www.researchgate.net/profile/S-Taft/publication/286243707_Ada_83_to_Ada_2012/links/56e02f1408aec4b3333d0201/Ada-83-to-Ada-2012.pdf (https://www.researchgate.net/profile/S-Taft/publication/286243707_Ada_83_to_Ada_2012/links/56e02f1408aec4b3333d0201/Ada-83-to-Ada-2012.pdf) Unfortunately I do not have access to the text of the actual presentation.

This may be a little controversial: It is now year 2022, and modern OSs have support for Unicode character set. Basically there is no reason to use plain ASCII character set, so the language could use some symbols defined in Unicode character set. To be clear here, I do not want to see a new APL either. :)

Ada now uses [ and ] but as syntax for array initialization:

https://learn.adacore.com/courses/whats-new-in-ada-2022/chapters/array_aggregates.html (https://learn.adacore.com/courses/whats-new-in-ada-2022/chapters/array_aggregates.html)

Hmm Ada also lets you specify the type of the subscript expression, interesting...
Title: Re: A microcontroller programming language
Post by: brucehoult on December 23, 2022, 02:27:46 am
Yes, all args are passed by ref. To achieve pass by value (that is, pass a ref to a temp copy) one just surrounds the arg in parens:

omfg. I am without words.

You just threw away a huge chunk of performance, as well as compatibility with all existing libraries and OS interfaces.

Ha! but not without expletives!

OK since you do want to discuss this with me, let me respond.

I'm referring to the general policy for arg passing, ultimately an implementation could do this in various ways (pass in a register for example). I'm not really referring to the implementation but to the semantics, I don't think I made that clear.

So "pass by reference" here is meant to indicate that the callee can change something in the datum that's been passed. That's the default convention, and if the the caller want's to prevent a callee from changing the value of an arg, they'd simply make that arg an expression with ( and ).

This leads to increased performance because unless the developer wants to pass by value, they can write the code and just let args pass by ref, meaning passing a bunch of structures, strings, arrays or combinations of these will entail nothing more than pushing a pointer onto the stack!

Ugh.

Code: [Select]
procedure foo(i : integer)
  var i : integer = 1;
  procedure bar(j : integer)
    i := i + 1;
    j := j + 1
  end
  bar(i);
  print(i)
end

What is printed?
Title: Re: A microcontroller programming language
Post by: DiTBho on December 23, 2022, 04:34:48 pm
I have a feeling this language will suck more than what (C?) is going to replace  :o :o :o
Title: Re: A microcontroller programming language
Post by: Kalvin on December 26, 2022, 01:06:42 pm
I have a feeling this language will suck more than what (C?) is going to replace  :o :o :o

It is very hard to create a new, useful programming language which would gain a wide user base.

As we have now over 70 years of experience in different kind of programming languages, the author should really take a deep look into the most popular languages used in embedded software design, and extract the best features from those languages into the new language, while avoiding pitfalls in the existing languages as much as possible.

This discussion thread has some very good set of features that could be provided by the new language.

Since implementing a programming language is a big task, the author might want to introduce the new language in smaller steps, starting from a simple core language, and implementing more advanced features in small steps. However, in order to avoid duct-tape syntax and grammar blunders, the author should have a fairly good and mature plan for how to implement the advanced syntax and grammar features as a whole in the future language releases.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 26, 2022, 02:01:08 pm
This discussion thread has some very good set of features that could be provided by the new language.

Seen nothing good, too much nonsense repeated over and over again, and the latest*hit seems to be written by someone who has never planned anything serious in his professional life.

Good show :popcorn:
Title: Re: A microcontroller programming language
Post by: DiTBho on December 26, 2022, 02:09:49 pm
the author should really take a deep look into

Frankly, the author should really get a job. I mean programming real stuff, from home automation to avionics. Only then can he come back here and talk. Otherwise, sugar crap is and will be the only main core of everything.

Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 26, 2022, 07:18:20 pm
the author should really take a deep look into

Frankly, the author should really get a job. I mean programming real stuff, from home automation to avionics. Only then can he come back here and talk. Otherwise, sugar crap is and will be the only main core of everything.

That was fun. ;D

To be fair, we don't know much about the OP's background and I wouldn't be so harsh about it, maybe he actually has accomplished a lot, we don't know. You may be surprised seeing how many professional software engineers 1/ have an obsession with designing a new language, 2/ have a very personal view of programming, and 3/ tend to focus on specific points that are likely stuff that has bitten them in the arse during their career.

So when they design a new programming language, they tend to come up with a pile of ad-hoc concepts and features, rather than take a step back and look at the big picture. Actually, most programming languages ever designed either fall into this category, or were designed in an academic context. In the third category, which is an exception, I would put languages such as Ada or Modula-3, which were designed by groups of professionals with definite goals, rather than individuals focusing on their own perceptions.

Note that, as we often discuss, this certainly has an impact on the intrisic qualities of a language, but little on its popularity. A number of languages that have become very popular were just a bunch of ad-hoc stuff stitched together. Sad but just how it goes.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 27, 2022, 01:54:35 am
Note that, as we often discuss, this certainly has an impact on the intrisic qualities of a language, but little on its popularity. A number of languages that have become very popular were just a bunch of ad-hoc stuff stitched together. Sad but just how it goes.
PHP (https://en.wikipedia.org/wiki/PHP) is perhaps the best example of this.

Considering that, why would anyone start designing a language from the ground up, starting with syntax or grammar, instead of examining how expressions and algorithms would be best expressed in the new language?

For example, if one intends the language to be used for linear algebra type stuff, then array slicing – which goes back to at least Fortran, so nothing new! – is particularly important.  If one intends the language to be used for server-side HTTP stuff, then efficient and effective I/O is paramount, as well as interfacing to external datastores (files, databases, external servers).  For the tasks microcontrollers (as opposed to embedded systems like SoCs) are used, latencies and overhead should be minimal.

For microcontrollers, we have many imperative languages, but very few event-based languages, even though many microcontrollers basically just respond to events, rather than follow imperative logic.  The imperative languages (C/C++, Ada/Spark, Forth, etc.) have decades of work behind them, so unless one creates a variant by only replacing the parts they believe are keeping developers back, starting from scratch like Go (https://en.wikipedia.org/wiki/Go_(programming_language)), Julia (https://en.wikipedia.org/wiki/Julia_(programming_language)), Rust (https://en.wikipedia.org/wiki/Rust_(programming_language)) did will involve at least a decade of development before it becomes comparable; this seems to apply regardless of what kind of programming language one is developing.

One intriguing approach would be to design a language better suited for event-based programming, suitable for use on a microcontroller.  (Can you even create an event-based language without a significant runtime?)  Things like interrupts do map very well to events, but I'm talking more about the internal code structure: instead of the code being based on an imperative sequence, it would be based on posting and consuming events.
Rather than syntax or grammar, one should first examine how one would implement typical tasks in such event-based code, and then compare the maintainability, robustness, and reliability of that code to the same implemented in one of the imperative languages.  How obvious is the event-based code, compared to the imperative code?  How many ways one can write a buggy version of the same without the bugs being obvious?

At the bottom, there is always the machine code implementation.  Languages with a significant runtime have their place (in desktop applications, mostly), but in a microcontroller, it seems like a waste of resources: using abstractions where abstractions are not needed, just to help unskilled programmers produce low-quality implementations.  So, a microcontroller programming language should be efficient wrt. resource use, and the abstractions used map cleanly (without complicated translation layers) to underlying hardware.  This means along with the example code implementation in the new language, one should carefully consider and show the expected machine code the new programming language code might compile to.  (Of course not exactly or perfectly; but approximately, showing how the entire high-level abstract concepts get mapped to the new programming language, and that eventually to the underlying machine code.  If you do need a significant runtime, say a bytecode interpreter or JIT compiler in between, you really need to drop the 'microcontroller' part out; most microcontrollers don't have the resources to support such.)
Title: Re: A microcontroller programming language
Post by: tggzzz on December 27, 2022, 11:44:29 am
One intriguing approach would be to design a language better suited for event-based programming, suitable for use on a microcontroller.  (Can you even create an event-based language without a significant runtime?)  Things like interrupts do map very well to events, but I'm talking more about the internal code structure: instead of the code being based on an imperative sequence, it would be based on posting and consuming events.

Rather than syntax or grammar, one should first examine how one would implement typical tasks in such event-based code, and then compare the maintainability, robustness, and reliability of that code to the same implemented in one of the imperative languages.  How obvious is the event-based code, compared to the imperative code?  How many ways one can write a buggy version of the same without the bugs being obvious?

Those are, IMNSHO, very sensible and important questions, and ones that do lead to practical advances in the field.

On example is xC, where there are no interrupts[1] because input events (in the form of messages from an I/O port or processor) are handled by a "superloop". There is no RTOS per se, because those benefits are achieved by hardware and multiple independent processors. It enable FPGA-like hard realtime guarantees. When "kicking the tyres" of the technology, I found it stunningly easy to use and without surprises.

There are also many event-based DSLs. While many are pretty scrotty hacks, ISTR one that was used in safety critical avionics. (?Kennedy Carter, kc.com? I came across it at the turn of the milennium!)

[1] a slight lie, but accurate to all intents and purposes
Title: Re: A microcontroller programming language
Post by: DiTBho on December 27, 2022, 01:11:27 pm
Can you even create an event-based language without a significant runtime?

Probably ... no  ;D
But it would be WOW!
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 27, 2022, 06:26:29 pm
Can you even create an event-based language without a significant runtime?

Probably ... no  ;D
But it would be WOW!

I think this could be achieved with a relatively small runtime.
Message passing schemes is one of my "favorite" topics in programming lately, and it could be applied to passing and handling events.
Sure some hardware support would help getting better performance, but even on a classic architecture with interrupts you could implement that.

I've implemented some of that with pthreads. Haven't done this for small targets yet, but it should be achievable. And achievable using almost any programming language, certainly C. Maybe with some assembly for the "runtime".
Integrating within the language itself being mostly syntax sugar coating. That has always been Wirth's opinion too. Anyone's free to differ.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 27, 2022, 07:59:42 pm
Can you even create an event-based language without a significant runtime?

Probably ... no  ;D
But it would be WOW!

Of course you can.

Since you have complete control of the machine, it isn't difficult, but you benefit from having tiny parts in the relevant machine code, to  swap stacks, avoid priority inversion, ISR exit/entry, and atomic mailboxes.

Have a look at how simple RTOSs are implemented. Ignore those that try to do everything in C, since you can't (unless you are prepared to have severe constraints with programming style).

Ignore POSIX; it is too heavyweight.

Title: Re: A microcontroller programming language
Post by: tggzzz on December 27, 2022, 08:13:30 pm
Can you even create an event-based language without a significant runtime?

Probably ... no  ;D
But it would be WOW!

I think this could be achieved with a relatively small runtime.
Message passing schemes is one of my "favorite" topics in programming lately, and it could be applied to passing and handling events.
Sure some hardware support would help getting better performance, but even on a classic architecture with interrupts you could implement that.

Exactly!

Message passing isn't well taught in traditional undergrad computing courses, for several reasons...
It isn't a computer science topic, cf lambda calculus, compilers, a standard algorithm.
It comes from hardware engineering.
Mainstream languages don't have it as a feature; it is relegated to libraries.

OTOH those that have to push hardware and software to achieve scaleable hetereogeneous long-lifetime high-complexity high-availability systems absolutely rely on message passing. Obvious examples are high performance computing (HPC traditionally stresses computing hardware and software so far that they break in unanticipated ways), telecoms, avionics, etc.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 27, 2022, 09:07:52 pm
Can you even create an event-based language without a significant runtime?

Probably ... no  ;D
But it would be WOW!

I think this could be achieved with a relatively small runtime.
I happen to agree.

One approach I use extensively in Linux programming is a binary heap of expiration times (timed events) with an associated array of event descriptions, managed by a dedicated pthread.  Given uniform random expiration times, adding or removing an event involves just e≃2.718 percolation events on average, so both adding and removing new events is quite lightweight and doesn't pollute the cache too much.   The event descriptions stay put, and contain an index to the heap; each entry in the heap is a tuple of the expiration time and an index to the description.  The descriptions vary by what the program does, but typically contain an expiration flag or a POSIX semaphore (sem_t).  To me, this shows that time-related events are not a problem even on small microcontrollers.

One realization I had when writing that post was that in the vast majority of cases, event handlers do not need to be re-entrant, and they either return nothing, or a flag indicating whether the event should not be propagated (or possibly postponed).  (I've written quite a bit of event-based code, from Macromedia Lingo to Javascript handlers, as well as GUIs using GTK, Qt, Tk, etc. toolkits.)  If this is considered in the internal ABI –– for example, making "functions" and "event handlers" ("procedures"?) have completely separate calling mechanisms –– it opens up a lot of possibilities.  For example, consider the case where event handlers have a separate "state context" or object they refer to (indirectly via a pointer register) to record state.  An event is passed in the registers, so that stack is not really used (except for the call chain return addresses and interrupts).
The translation layer (for example, interrupt handler preamble) is very thin, and minimal amount of RAM is needed.
To dispatch an event, you need to know the event handler, its state context, and what to pass in the registers.

The largest part of the runtime would be the superloop event dispatcher.  Depending on the internals, it likely would be quite simple; possibly quite similar to the binary timeout heap thread I mentioned above, augmented with a priority queue of events to be dispatched as soon as possible.

Thing is, what would the actual code look like?  My own opinion is colored by my experience, so I have strong opinions on some facets, but I am absolutely certain I alone would not end up with the best possible syntax.

Also, is a single global event queue sufficient, or are there useful patterns where multiple independent event queues are needed?

Lots of questions to be explored, for sure; and a lot to gain, compared to current programming languages used on microcontrollers, for sure.
Which makes me so darned frustrated to see someone concentrate on syntax and grammar first, before even discussing the target programming model or paradigm.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 27, 2022, 09:45:59 pm
Have a look at how simple RTOSs are implemented. Ignore those that try to do everything in C, since you can't (unless you are prepared to have severe constraints with programming style).

Yup, I have already ported ucOS/2 and XINU to m68k (Gcc) and MIPS5++ (my-c).

my-c adds support for tr-mem, which is useful to implement semaphores and message passing between cores and cops, but I am not prepared to have more constraints with programming style, thus, in both cases specific modules written in assembly cover all the machine-specific code.

I cannot add any event-driven feature to my-c, anyway.
Only libraries.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 27, 2022, 10:04:40 pm
I'm not particularly familiar with ucos, but it seems to be much more than necessary for an event driven kernel (e.g. a GUI, FTP, DHCP etc).

But at least you realise your statement "Probably ... no  ;D But it would be WOW!" is unduly pessimistic.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 27, 2022, 10:08:24 pm
Message passing schemes is one of my "favorite" topics in programming lately, and it could be applied to passing and handling events.
Message passing isn't well taught in traditional undergrad computing courses, for several reasons...
It isn't a computer science topic, cf lambda calculus, compilers, a standard algorithm.
It comes from hardware engineering.
Mainstream languages don't have it as a feature; it is relegated to libraries.
I've used MPI a lot in both C and Fortran, but my use cases are HPC and distributed computing, and the messages contain shared simulation data.
I'm also familiar with message mailbox techniques used in e.g. Linux kernel drivers, but haven't really explored microkernels using message passing.
Simply put, my own experience in message passing is too one-sided –– distributed data as messages –– to know much about how one could treat events as messages at the low (machine code) level in an efficient manner.  I am sure it can be done, I just don't know exactly how.

(In MPI, each message is identified by a tuple (sender, tag), and messages with different identifying tuples can be received in any order.  This makes asynchronous/nonblocking message passing (as in MPI_Isend() and MPI_Irecv()) especially powerful.  The per-process runtime in OpenMPI is a helper thread, which coordinates the messaging between processes, and is surprisingly lightweight.)

Simple events like timeouts and interrupts do seem quite straightforward, but when we look at cases where the message involves significant amounts of data –– say, a DMA completion interrupt –– we'll likely find we need to also consider zero-copy (https://en.wikipedia.org/wiki/Zero-copy) techniques; not so much for speed or efficiency, but because RAM and buffer space is quite limited in microcontrollers.

One possibility for handling zero-copy buffer events is to have "acknowledged events": When the buffer is full, the related event is dispatched.  The buffer is "owned" by the event handler, until it acknowledges the buffer-full event by a corresponding buffer-now-free event.  Obviously, there is nothing special in such acknowledgements, it is just a programming paradigm.  We just know from existing programming languages that the approach must be consistent and useful, or we'll end up with footguns like gets() in C.  This is also why I believe development through real-world examples/problems is the way to go.
(I am fully aware that I just described a completely asynchronous/non-blocking I/O or buffer-passing mechanism.  This was deliberate, as we really do need this to make better MCU firmware.  Less buffer-bloat, and so on.)

If I were to try and develop such a programming language, I would start by creating a simple IP stack and an example application, perhaps a HTTP server displaying a static page (with GET and HEAD support), in this new language.  The language, the IP stack implementation, and the low-level machine code generated (roughly – no need to have an actual compiler, just the rough intent suffices, perhaps call ABI on a specific architecture), would all be developed in parallel.  One could examine its readability and maintainability by giving the language spec and a snippet of code with a bug in it (something the compiler would not complain about; a thinko rather than a typo) to a suitable victim test person, describing the effect of the bug, and asking them to find the bug.  If the person finds the bug without having to consult the language spec beyond the introduction, we'd be on the right track.

Things like keywords or reserved words, operators, exact syntex, and so on, are secondary, because the point is to develop a language to effectively express the concepts.  You could even have antagonistic teams, finding ways of obfuscating code and hiding bugs in it, to arrive at the most robust syntax.

That the large organizations like Google do not seem to grok this at all, and instead focus on churning out yet another imperative object-oriented language after another, really depresses me.  So, perhaps I'm a bit hard on OP for going down the exact same unfruitful road, but I definitely have a reason to here, it's not just "my opinion" here.  The entire method is at fault, and a completely different design approach should be taken.
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 27, 2022, 10:18:36 pm
I'm not particularly familiar with ucos, but it seems to be much more than necessary for an event driven kernel (e.g. a GUI, FTP, DHCP etc).

But at least you realise your statement "Probably ... no  ;D But it would be WOW!" is unduly pessimistic.

I've written a small preemptive scheduler (priority-based + per-task tick period) for my RISC-V core, which would lend itself well to event-driven stuff. One C file and one assembly file. Total object code is about 5KB with optimizations enabled. There is no specific feature for passing messages at this point, but it wouldn't be too hard to do and would probably take no more than 1 extra KB or so. I've put this on hold, but I was considering what kind of hardware blocks I could add to my core to make this more efficient. Sure there is context switching which takes a few tens of cycles. Could be avoided if I didn't make this preemptive. In order to avoid latency due to cache issues (on systems with a cache), one can define a core-coupled area of RAM dedicated to context saving/dedicated stacks/messages, so it doesn't have to go through any cache.


Title: Re: A microcontroller programming language
Post by: tggzzz on December 27, 2022, 10:33:41 pm
I'm not particularly familiar with ucos, but it seems to be much more than necessary for an event driven kernel (e.g. a GUI, FTP, DHCP etc).

But at least you realise your statement "Probably ... no  ;D But it would be WOW!" is unduly pessimistic.

I've written a small preemptive scheduler (priority-based + per-task tick period) for my RISC-V core, which would lend itself well to event-driven stuff. One C file and one assembly file. Total object code is about 5KB with optimizations enabled. There is no specific feature for passing messages at this point, but it wouldn't be too hard to do and would probably take no more than 1 extra KB or so. I've put this on hold, but I was considering what kind of hardware blocks I could add to my core to make this more efficient. Sure there is context switching which takes a few tens of cycles. Could be avoided if I didn't make this preemptive. In order to avoid latency due to cache issues (on systems with a cache), one can define a core-coupled area of RAM dedicated to context saving/dedicated stacks/messages, so it doesn't have to go through any cache.

Sounds right.

I'm not a fan of pre-emptive event processing. I prefer a superloop where an event is consumed and the actions run to completion before processing the next action. The action can be short duration computation/io/etc, and usually concludes by generating another event that moves the FSM or FSMs into another state. That's sometimes called the half-sync half-async design pattern; where the "sync" computation is followed by the "async" event.

If the processing is going to take too long, then encapsulate it in a separate start.event plus completion event, and modify the FSM accordingly.

Caches kill hard real-time guarantees stone dead; that's why xC on xCORE processors avoid them as well as interrupts.
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 27, 2022, 10:48:32 pm
I understand your point about preemption, but cooperative multitasking can get messy pretty fast, and the cost of keeping context for each task can be every bit as expensive, if not more in the end, than just saving and restoring all registers. You end up with pretty intricate state machines. I've written a FAT32/exFAT library in this cooperative way (in order to have no blocking call whatsoever), and it was a real PITA.

And if you stick to messages/events as synchronization means, then a lot of the hard stuff about synchronization becomes a lot easier.

Meeting deadlines is also easier in general with preemption, as long as priorities and tick periods are handled appropriately (so not in the way general-purpose schedulers usually do.)
Title: Re: A microcontroller programming language
Post by: tggzzz on December 27, 2022, 11:05:48 pm
I understand your point about preemption, but cooperative multitasking can get messy pretty fast, and the cost of keeping context for each task can be every bit as expensive, if not more in the end, than just saving and restoring all registers. You end up with pretty intricate state machines. I've written a FAT32/exFAT library in this cooperative way (in order to have no blocking call whatsoever), and it was a real PITA.

I don't disagree :(

The ameliorations are good taste and clear thinking, neither of which are in common supply. Even then, sometimes the application doesn't suit the design pattern; have many tools in your toolbox, and pull out the right one for the job.

Quote
And if you stick to messages/events as synchronization means, then a lot of the hard stuff about synchronization becomes a lot easier.

Exactly. And given the strong similarities between timeouts, inputs arriving, outputs completing, messages to/from other processes/cores, there seems little reason not to have one mechanism for the whole lot.

Quote
Meeting deadlines is also easier in general with preemption, as long as priorities and tick periods are handled appropriately (so not in the way general-purpose schedulers usually do.)

My tastes are otherwise, but no doubt I could be convinced in some cases.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 28, 2022, 12:55:55 am
But at least you realise your statement "Probably ... no  ;D But it would be WOW!" is unduly pessimistic.

I mean I don't think I am good enough to make my-c an event-oriented language.
I can use my-c to write event-oriented software, like I can use C, but there is no improvement between them, except my-c has tr-mem support, so if your hardware has tr-mem (POWER10), it can simplify some inner message passing mechanisms.

Mailboxes can have support as well, but it hasn't been designed to serve any event-oriented paradigm, so it's nothing new, nothing revolutionary (like xc), you still have to write a lot of assembly. Just a bit less.

Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 28, 2022, 07:23:48 am
It is funny how it all really boils down to RAM use, on the smallest microcontrollers.

Even on 32-bitters, a lot of time and effort is spent to find out exactly how much RAM to reserve for each task stack.
Even in Linux, the default stack size is so large (2-8 megabytes, depending on the 32-bit architectures) that it is the first limiting factor on the number of parallel threads a process can have.  (On most 32-bit architectures in Linux, 16384 bytes is fine for non-recursive threads.)

The very first things I'd drop from an ABI is the concept of a function stack frame.  The less stack is used, the more you can do with limited RAM.  (Anyone who suggests a red zone (https://en.wikipedia.org/wiki/Red_zone_(computing)) should be shot outright.  That damn thing makes any kind of stack use tracking via the stack pointer unreliable –– useless, really.)

It also means it is best to pass data by value in registers, so that data already in registers does not have to be written to RAM just to call a function.
Passing data by reference is an utter pain.

Many microcontroller tasks are easier to describe in terms of generators and coroutines.  For example, a COBS stuffer/unstuffer can be implemented as a simple state machine FIFO.  By their very nature, they're non-reentrant.  So, if a language –– be it event-based or imperative –– supports multiple forms of "functions", one could be a closure-like coroutine/generator/producer-consumer, which uses a state object to record its own state, and stack only for calling functions; "consume"/"produce" ("next"/"yield") would only be allowed in the function itself, and not in any functions it calls.  This way, one would not need to swap stacks to call these, since they may not use stack across a "consume"/"produce" event.  Also, the ABI can define which registers are saved/lost across "consume"/"produce", further reducing the cost of using such coroutines.
I haven't yet discovered a really good way to express such sequence-processing "functions", but they definitely need to be handled differently to ordinary functions.  (Even Python 3 is still struggling a bit wrt. generator functions; see e.g. yield expressions (https://docs.python.org/3/reference/expressions.html#yieldexpr).)

The above also means that recording programmer intent, similar to how one can use function attributes with GCC and Clang, so that the compiler can enforce the rules when compiling the code, is important.  I personally am having trouble thinking up good descriptive names for the things, much less worry about how to syntactically express the darn things..  ::)
Title: Re: A microcontroller programming language
Post by: Kalvin on December 28, 2022, 08:00:04 am
From time to time I have been missing a [C] preprocessor that would be able to generate hygienic macros, and would be coupled to the compiler more tightly so that the macros could determine and check the types of the macro parameters. In that way it would be possible to extend the core language, and for example create framework for implementing events, state machines, patterns, and templates without making the core language too complex.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 28, 2022, 10:48:37 am
...
which uses a state object to record its own state, and stack only for calling functions;
...

That's a very good technique, especially where there are many independent FSMs running simultaneously, e.g. telecom phone calls, or web connections.

Have a single FIFO containing events yet to be processed. Have one "worker thread" per core/processor dedicated to consuming an event, doing the actions, and creating another event. Each worker thread sucks the next event from the FIFO. Doesn't require co-routines. It all leads to good scaleable high performance applications.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 28, 2022, 10:59:32 am
From time to time I have been missing a [C] preprocessor that would be able to generate hygienic macros, and would be coupled to the compiler more tightly so that the macros could determine and check the types of the macro parameters. In that way it would be possible to extend the core language, and for example create framework for implementing events, state machines, patterns, and templates without making the core language too complex.

I hate that kind of thing :)

Need to read/parse all the codebase to understand what's happening.

Templating languages end up being Turing complete monsters - like the C++ monstrosity did by accident. https://en.wikibooks.org/wiki/C%2B%2B_Programming/Templates/Template_Meta-Programming#History_of_TMP

There are several excellent design patterns for implementing FSMs in conventional languages. I haven't found a reason not to use them.
Title: Re: A microcontroller programming language
Post by: Kalvin on December 28, 2022, 11:02:27 am
...
which uses a state object to record its own state, and stack only for calling functions;
...

That's a very good technique, especially where there are many independent FSMs running simultaneously, e.g. telecom phone calls, or web connections.

Have a single FIFO containing events yet to be processed. Have one "worker thread" per core/processor dedicated to consuming an event, doing the actions, and creating another event. Each worker thread sucks the next event from the FIFO. Doesn't require co-routines. It all leads to good scaleable high performance applications.

Miro Samek's freely available ebook Practical UML Statecharts in C/C++, 2nd Ed Event-Driven Programming for Embedded Systems
https://www.state-machine.com/psicc2 (https://www.state-machine.com/psicc2)
describes an Active Object pattern decoupling the event producers and consumers:
https://www.state-machine.com/active-object (https://www.state-machine.com/active-object)

This Active Object-pattern can be used as an alternative to / with the system-wide event-queue.
Title: Re: A microcontroller programming language
Post by: Kalvin on December 28, 2022, 11:09:47 am
I hate that kind of thing :)

I know that this idea is a bit controversial. Extending the language with an intelligent preprocessor might help when implementing new frameworks and expressing FSMs etc.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 28, 2022, 11:11:31 am
It also means it is best to pass data by value in registers, so that data already in registers does not have to be written to RAM just to call a function.
Passing data by reference is an utter pain.

Yup. The only good use is for objects, so you pass the pointer to a large structure.

kind of f(p_obj, value);

p_obj points to a struct of { methods, properties, data-context ... }
value is passed by register or by stack.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 28, 2022, 11:19:20 am
[C] preprocessor that would be able to generate hygienic macros

I hate that kind of thing :)

me too, indeed the preprocessor was the first thing I removed from my-c.

oh, its implementation also sucks in gcc, at least from gcc-v2.95 to gcc-v3.3.*
Yesterday I prepared a builder, and I had to *hack* gcc, because it stops working in modern Linux ecosystems.
Code: [Select]
macmini-G4 /projects/devchain-baremetal-2022 # ./mybuild info all
m6811-elf, binutils v2.23, gcc v3.3.6-s12x
m88k-coff, binutils v2.16.1, gcc v2.95.3
m68k-elf, binutils v2.34, gcc v4.1.2-core
mips-elf, binutils v2.24, gcc v4.1.2-core
mips64-elf, binutils v2.24, gcc v4.1.2-core
powerpc-elf, binutils v2.24, gcc v4.1.2-core
powerpc-eabi, binutils v2.24, gcc v4.1.2-core
arm-eabi, binutils v2.24, gcc v4.1.2-core
sh2-elf, binutils v2.24, gcc v4.1.2-core
bugged parts: cpp0, cpp, collect.

Whatever you do, don't do it that way  ;D
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 28, 2022, 11:54:20 am
...
which uses a state object to record its own state, and stack only for calling functions;
...

That's a very good technique, especially where there are many independent FSMs running simultaneously, e.g. telecom phone calls, or web connections.

Have a single FIFO containing events yet to be processed. Have one "worker thread" per core/processor dedicated to consuming an event, doing the actions, and creating another event. Each worker thread sucks the next event from the FIFO. Doesn't require co-routines. It all leads to good scaleable high performance applications.

Miro Samek's freely available ebook Practical UML Statecharts in C/C++, 2nd Ed Event-Driven Programming for Embedded Systems
https://www.state-machine.com/psicc2 (https://www.state-machine.com/psicc2)
describes an Active Object pattern decoupling the event producers and consumers:
https://www.state-machine.com/active-object (https://www.state-machine.com/active-object)

This Active Object-pattern can be used as an alternative to / with the system-wide event-queue.

If you look at efficient service daemons in Linux and Unix systems, nonblocking I/O using select()/poll() does something extremely similar:
Each connection is their own state object, and there is a simple "event" loop, handling all descriptors that are readable/writable in turn, and otherwise blocking in the select()/poll() call.

Expressing this pattern in a better way would be nice.  This is also what I meant by having more than one event loop.

It also means it is best to pass data by value in registers, so that data already in registers does not have to be written to RAM just to call a function.
Passing data by reference is an utter pain.
Yup. The only good use is for objects, so you pass the pointer to a large structure.
Or to an array, yes.

I do prefer the C way of explicitly defining references (pointers) as separate types, and believe the C++ way of letting the function signature describe whether it takes a value or a reference to a value is likelier to lead to programmer errors.  The latter seems like added complexity just to save the programmer from having to write an extra address-of operator in each call.

Python passes by assignment, which basically means that ints, floats, strs, tuples, and other immutable types are passed by value, but dictionaries, lists, sets, and mutable types and objects are passed by reference.  It doesn't seem to be a problem for Python programmers, which indicates to me that the passing scheme (by value, by reference, or by assignment like in Python) doesn't seem to significantly affect the code; so simplicity should be favoured for the reasons I already outlined.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 28, 2022, 02:34:24 pm
If you look at efficient service daemons in Linux and Unix systems, nonblocking I/O using select()/poll() does something extremely similar:
Each connection is their own state object, and there is a simple "event" loop, handling all descriptors that are readable/writable in turn, and otherwise blocking in the select()/poll() call.

Expressing this pattern in a better way would be nice.  This is also what I meant by having more than one event loop.

If we concentrate on the main "worker" engine doing most of the processing, and ignore separate event loops which create events....

Is there any performance value in having more than one event loop per core?
Is there any simplicity/clarity value in having more than one event loop per core?

Quote
Python passes by assignment, which basically means that ints, floats, strs, tuples, and other immutable types are passed by value, but dictionaries, lists, sets, and mutable types and objects are passed by reference.  It doesn't seem to be a problem for Python programmers, which indicates to me that the passing scheme (by value, by reference, or by assignment like in Python) doesn't seem to significantly affect the code; so simplicity should be favoured for the reasons I already outlined.

I tend to ignore Python where high performance and scalability is required, since despite having "threads" it is "crippled" by its Global Interpreter Lock. Ditto Ruby. That may be unjust, but...
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 28, 2022, 08:20:15 pm
But at least you realise your statement "Probably ... no  ;D But it would be WOW!" is unduly pessimistic.

I mean I don't think I am good enough to make my-c an event-oriented language.

Question being, as I mentioned earlier, do you really need to design a specific language for this? Apart from the sugar coating?

One point that has been raised and would indeed require to be embedded in the language itself would be the possibility of not using a stack (although I have mixed feelings about that), which you can't usually control with existing languages (unless you use assembly directly.)

Admittedly, another point would be static analysis though. If the language is designed to be "event-oriented", then it becomes possible to statically analyze event handling and prove some level of correctness, while it's a lost cause with classic imperative languages.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 29, 2022, 12:49:36 pm
do you really need to design a specific language for this? Apart from the sugar coating?

I think the right question is: how can I reduce the amount of - and tricks behind the stage - aka assembly code - I have to write to serve events-programming in a language?

Title: Re: A microcontroller programming language
Post by: tggzzz on December 29, 2022, 01:42:50 pm
do you really need to design a specific language for this? Apart from the sugar coating?

I think the right question is: how can I reduce the amount of - and tricks behind the stage - aka assembly code - I have to write to serve events-programming in a language?

No more than is required for an RTOS with cooperative scheduling, preferably plus a keyword to indicate waiting for any of several events. I know that since I used one such environment in C on a Z80, 40 years ago. So that was K&R C as documented in the two available books.

For a suitable keyword, see the xC "select" statement as I outlined earlier - i.e. a switch where the cases are events that are awaited.

You could make it arbitrarily complicated, of course, but that would be inelegant (just like C++ :) ).
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 29, 2022, 11:04:35 pm
If we concentrate on the main "worker" engine doing most of the processing, and ignore separate event loops which create events....

Is there any performance value in having more than one event loop per core?
Is there any simplicity/clarity value in having more than one event loop per core?
Excellent questions.  I don't know the answers, and the only way to find out I know of is to look at actual practical implementations.

I tend to ignore Python where high performance and scalability is required, since despite having "threads" it is "crippled" by its Global Interpreter Lock. Ditto Ruby. That may be unjust, but...
Oh, I only meant wrt. syntax: that whether parameters are passed by value or by reference (or by assignment, a "mix" of the two, as in Python), does not seem to affect the number of bugs in the code.

In my opinion, this means that because at the hardware level it makes a HUGE difference, passing by value being much, MUCH more efficient, passing by value is the superior approach for low-level languages.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 30, 2022, 12:15:40 am
If we concentrate on the main "worker" engine doing most of the processing, and ignore separate event loops which create events....

Is there any performance value in having more than one event loop per core?
Is there any simplicity/clarity value in having more than one event loop per core?
Excellent questions.  I don't know the answers, and the only way to find out I know of is to look at actual practical implementations.

Well, I surprised everybody by the performance of a half-sync half-async telecom event processor running on a Sun Niagara T1 with about one worker thread per core. "About" because I left a few cores for the GC and and input events and o/s stuff. It helped that the T.series processors have/had 64 SMT cores, each running at DRAM speed with no caches :)

As for implementations in the embedded niche, look at the xCORE processors architecture and how it is used with xC.
Title: Re: A microcontroller programming language
Post by: MIS42N on December 30, 2022, 01:12:18 am
In my opinion, this means that because at the hardware level it makes a HUGE difference, passing by value being much, MUCH more efficient, passing by value is the superior approach for low-level languages.
Don't you need both? Although I write assembler, I would think passing data in a higher level language would be similar. I have arithmetic stack operations, the two relevant subroutines to get stuff on the stack are:
Code: [Select]
CALL PushLit
12345
and
Code: [Select]
CALL PushU2
16bitUnsigned ; pointer to the value
So both pass by value and pass by reference.

For generating output the relevant subroutines are
Code: [Select]
CALL PrintChar ; value is in a registerand
Code: [Select]
CALL PrintStr
PointToString

Although passing by value is efficient in terms of CPU cycles, it isn't the whole picture. Passing by reference has CPU overhead but less instruction memory. Sometimes memory is the constraint. I don't think one can say one method is superior to the other, they both have advantages.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 30, 2022, 04:10:13 am
In my opinion, this means that because at the hardware level it makes a HUGE difference, passing by value being much, MUCH more efficient, passing by value is the superior approach for low-level languages.
Don't you need both?
If your language has a concept of pointer, then by constructing a pointer to an object and passing that pointer by value will give you the effects of passing by reference.

It was OP who suggested passing everything by reference.  This means that to pass a variable, one would need to store that variable in RAM first, before calling a function.

Comparing the SysV ABI (application binary interface, at the hardware level, assembly language) on x86-64 to most common x86 ABIs shows that passing by value, using registers to store values, and returning up to two 64-bit words, yields superior performance (regardless of language) because it minimises call overhead.

Although passing by value is efficient in terms of CPU cycles, it isn't the whole picture. Passing by reference has CPU overhead but less instruction memory.
No, that claim does not seem to be supported by practical evidence.

There are several calling conventions (https://en.wikipedia.org/wiki/Calling_convention#Different_calling_conventions) on x86 and x86-64, some that pass parameters on the stack, and some that pass parameters in registers.  If your claim was true, then passing parameters on stack would yield shorter functions than passing parameters in registers.  The opposite tends to be true.  Passing parameters in registers is also significantly faster on both architectures, but that difference is only visible on hardware that has caches and memory access slower than register access.

(If we consider passing parameters in registers by reference instead of by value, then the difference is significant: passing by reference involves an extra indirection.)

There are 8-bit architectures that have very few registers and were designed to pass function arguments on the stack, but they're quite difficult to optimize code for.  Most hardware today, from AVRs to ARMs to RISCV, have many general-purpose registers, so targeting those makes more sense to me anyway.
Title: Re: A microcontroller programming language
Post by: MIS42N on December 30, 2022, 08:43:00 am
If your language has a concept of pointer, then by constructing a pointer to an object and passing that pointer by value will give you the effects of passing by reference.
You are right. If you take a pointer to be a value, then I pass everything by value.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 30, 2022, 10:21:09 am
There are 8-bit architectures that have very few registers and were designed to pass function arguments on the stack, but they're quite difficult to optimize code for.  Most hardware today, from AVRs to ARMs to RISCV, have many general-purpose registers, so targeting those makes more sense to me anyway.

Not always true...

If you match main memory speed to core speed, then
That was demonstrated very clearly with the Sun UltrasSPARC Niagara T series processors, which had up to 128 "cores" (and used them effectively!) in 2005-2011, when x86-64 has <8 cores.

It is also the case in the xCORE devices, which have 32 cores for ~£25 (one off price at DigiKey).

Of course while such processors have superb aggregate throughput, their single-thread throughput is comparatively anaemic. But for "embarassingly parallel" servers and embedded applications, that's a good trade off.
Title: Re: A microcontroller programming language
Post by: DiTBho on December 30, 2022, 10:58:16 am
There are 8-bit architectures that have very few registers and were designed to pass function arguments on the stack

from 6800, to 6809, including 68hc11 ;D
Title: Re: A microcontroller programming language
Post by: DiTBho on December 30, 2022, 11:04:03 am
Sun UltrasSPARC

My reference is POWER10(1). With built-in tr-mem.
tr-mem costs silicon, thus money to us, POWER10 is *VERY* expensive, but ... I personally think it's worth it.

I mean, you'd better use silicon for tr-mem rather than for sliding registers window, plus complex out-of-order-reordering load/store units (that goes crazy, very crazy with specific pipeline instructions, which costs more silicon, more complexity in both hardware and software sides)


edit:
tr-mem, removed in Power ISA >= v.3.1
Title: Re: A microcontroller programming language
Post by: tggzzz on December 30, 2022, 11:13:47 am
Sun UltrasSPARC

My reference is POWER10. With built-in tr-mem.
tr-mem costs silicon, thus money to us, POWER10 is *VERY* expensive, but ... I personally think it's worth it.

I mean, you'd better use silicon for tr-mem rather than for sliding registers window, plus complex out-of-order-reordering load/store units (that goes crazy, very crazy with specific pipeline instructions, which costs more silicon, more complexity in both hardware and software sides)

One fundamental problem with transactional memory is that doesn't scale. Sooner or later too much time is eaten up by transaction-related activity. That was clearly demonstrated with the cache snooping on Athlon system: it worked fine with 4 chips, but despite best attempts it never worked well above that.

Another fundamental problem with transactional memory is what happens when a transactions are used inappropriately or a transaction fails to complete. That can be due to programming errors, memory errors, processor errors etc. Even if timeouts are implemented and work, how much unrelated activity is screwed until then?
Title: Re: A microcontroller programming language
Post by: DiTBho on December 30, 2022, 11:14:46 am
Comparing the SysV ABI (application binary interface, at the hardware level, assembly language) on x86-64 to most common x86 ABIs shows that passing by value, using registers to store values, and returning up to two 64-bit words, yields superior performance (regardless of language) because it minimises call overhead.

It's the same with PowerPC(1) and MIPS(2). I can confirm it's largely used (even by Windriver's and Green Hills(1)' toolchains) because it minimises call overhead.


(1) { C, C++, Ada } Optimizing Compilers - Green Hills.
(2) Microsoft C/C++ Optimizing Compilers for Pulsar PDA devices (2000s, WindowsCE-1.*)
Title: Re: A microcontroller programming language
Post by: DiTBho on December 30, 2022, 11:38:52 am
One fundamental problem with transactional memory is that doesn't scale
[...]
it worked fine with 4 chips, but despite best attempts it never worked well above that.

ummm, talking about my MIPS5++ experimental board, the CPU module has only 4 cores attached to a single tr-mem.

POWER10 requires a special Talos mainboard, different from the the first modern (post-2013) you can buy for POWER9 workstations. and servers, and the last POWER9 TL2WK2 comes with Two 4-core IBM POWER9 v2 CPUs.

Perhaps *only 4 cores* per package are too few. I need a bigger workstation  :-//
Title: Re: A microcontroller programming language
Post by: tggzzz on December 30, 2022, 12:13:57 pm
One fundamental problem with transactional memory is that doesn't scale
[...]
it worked fine with 4 chips, but despite best attempts it never worked well above that.

ummm, talking about my MIPS5++ experimental board, the CPU module has only 4 cores attached to a single tr-mem.

POWER10 requires a special Talos mainboard, different from the the first modern (post-2013) you can buy for POWER9 workstations. and servers, and the last POWER9 TL2WK2 comes with Two 4-core IBM POWER9 v2 CPUs.

Perhaps *only 4 cores* per package are too few. I need a bigger workstation  :-//

Where the scalability cliff is depends on many fine details; there are no simple extrapolations. Nonetheless, the cliff is there, somewhere.

What about this other point?...

Another fundamental problem with transactional memory is what happens when a transactions are used inappropriately or a transaction fails to complete. That can be due to programming errors, memory errors, processor errors etc. Even if timeouts are implemented and work, how much unrelated activity is screwed until then?
Title: Re: A microcontroller programming language
Post by: DiTBho on December 30, 2022, 01:13:19 pm
Another fundamental problem with transactional memory is what happens when a transactions are used inappropriately or a transaction fails to complete. That can be due to programming errors, memory errors, processor errors etc. Even if timeouts are implemented and work, how much unrelated activity is screwed until then?

Talking about the MIPS5++ cpu board installed on the MIPS.inc Atlas board, the tr-mem is built-in the CPU module.

my-c supports native tr-mem, it's a true type, with a defined behavior and memory model. This "saves" from misuse.

(well, with "save", I mean ...  necessary, not strictly sufficient, but decently sufficient if you don't do too weird things)

I followed all the points reported in the CPU documentation, and I did my best to achieve these points

At the moment the my-c policy for tr-mem failed transaction is:

           do nothing if it fails, let's exceptions do their resume job
                      (unless it's a too severe hw failure)
           abort and try again (it allows non-blocking mode).
           and if, and only if, it's too severe  (when voter reacts it's for a "severe hw failure"),
           propagate "hw severe failure" to the CPU port, wait4ever, disable channel and halt

This way you are guaranteed that a transaction will either be a success or a failure, never nothing in the middle, never nothing undefined.



(1) damaged hw is detected by two methods
- voter periodically cbit
- random internal cbit

cbit = continuous integrated tests. Sort of..."what is the result of (0xff + 0x01)|size32bit? ... I expect 0x100, right? no? your ALU is bad!!!". Done by machine exceptions.
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 30, 2022, 07:27:37 pm
Intel has added support for transactional memory with their TSX extension: https://en.wikipedia.org/wiki/Transactional_Synchronization_Extensions

It was added in Xeon processors years ago but was apparently defective, and the latest working version of TSX is in the current Skylake series. I don't know if any software/OS makes use of this extension yet.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 31, 2022, 12:47:23 am
There are 8-bit architectures that have very few registers and were designed to pass function arguments on the stack, but they're quite difficult to optimize code for.  Most hardware today, from AVRs to ARMs to RISCV, have many general-purpose registers, so targeting those makes more sense to me anyway.
Not always true...
Well, I do think 12 general purpose registers (xCORE-200 XS2 ISA (PDF) (https://www.xmos.ai/download/xCORE-200:-The-XMOS-XS2-Architecture-%28ISA%29%281.1%29.pdf)) is plenty! ^-^

But, sure, there are exceptions to any rule of thumb.  XCore is definitely one.

One optimization difficulty is how data has to be shuffled around between function calls.  When memory access is equally fast as register access –– so it doesn't matter whether you shuffle data in registers, or between registers and memory ––, then optimizing such shuffling becomes simple.

However, even in the xCORE XS2 ISA, arithmetic and logical operations are done in registers, with any of the 12 general purpose registers used as source and destination registers.  A function that does say additions, multiplications, and divisions between a few values, can do so directly if the values are passed in registers.  If the values are passed on stack, they have to be loaded from the stack before they're operated on.  This is the overhead one can avoid by passing by value in registers, with passing by reference implemented via pointers.

That was demonstrated very clearly with the Sun UltrasSPARC Niagara T series processors, which had up to 128 "cores" (and used them effectively!) in 2005-2011, when x86-64 has <8 cores.
Yep.  I personally like the idea of asymmetric multiprocessing a lot, and would love to play with such hardware.  Alas, the only ones I have are mixed Cortex-A cores (ARM big.LITTLE (https://en.wikipedia.org/wiki/ARM_big.LITTLE)).

We can see from the increasing complexity of peripherals on even cheap ARM Cortex-M microcontrollers, and things like Raspberry Pi Pico/RP2040 (and of course TI AM335x with its PRUs) that we're slowly going that way anyway.

Personally, I'd love to have tiny programmable cores with just a basic ALU – addition, subtraction, comparison, and bit operations, with just a couple of run-time registers – and access to reading and writing from the main memory, and the ability to raise an interrupt in the main core.  Heck, all the buses I use could be implemented with one.

I looked at XEF216-512-TQ128-C20A (23.16€ in singles at Digikey, in stock), and the problem I have with it is that it is too powerful! :o
I fully understand why XMOS developed xC, a variant of C with additions for tile computing; something like this would be perfect for developing a new event-based programming language, since it already has the hardware support for it.

For now, however, I have my own sights set much lower: replacing the standard C library with something more suitable for my needs, to be used in a mixed freestanding C/C++ environment.  This is well defined in the standards, and there are more than one compiler I can use it with (specifically, GCC and Clang on different architectures, including 8-bit (AVR), 32-bit (ARM, x86), and 64-bit (x86-64, Aarch64/ARM64) ones).
Title: Re: A microcontroller programming language
Post by: MIS42N on December 31, 2022, 01:00:37 am
Is this is going off topic. Xeon processors, MIPS, Athlon, power10?

I'm thinking a microcontroller has less CPU grunt but is augmented by a few onboard PWMs, extra timers, voltage references, comparators, ADC and DAC ability, USARTs, onboard USB, I2C, SPI, WDT, writable permanent memory, sleep modes and commonly low powered so it can be run off battery power. There may be some offloading of functions such as DMA and gated timers. And commonly just 1 CPU.

How do you then optimize for different architectures - AVR with multiple registers but transfers to memory are load and store - PIC with one working register but instructions like INCFSZ (increment memory and skip if zero) that doesn't touch the register at all. And maybe optimize for speed or optimize for minimum program instructions but in the same program.

Or was 'microcontroller' relegated SYS$BLACKHOLE (sorry - my VMS background creeping in).

Perhaps the best course is to wait for Sherlock to produce the language, and see if what benefits (if any) it brings. And with something to work with, then make comment to improve (if feasible).


Title: Re: A microcontroller programming language
Post by: DiTBho on December 31, 2022, 02:04:18 am
If the values are passed on stack, they have to be loaded from the stack before they're operated on.  This is the overhead one can avoid by passing by value in registers, with passing by reference implemented via pointers.

a little hw-trick I see on the MIPS5++ CPU module on the Atlas board: it uses 128Kbyte synchronous ram which always takes 1 clock (@80Mhz) to complete every read/write operation. It's kseg mapped (not cached).

Nice trick! This way the stack is as fast as registers  :o :o :o
Title: Re: A microcontroller programming language
Post by: DiTBho on December 31, 2022, 02:09:31 am
replacing the standard C library with something more suitable for my needs

That's my plan for 2023 for m68k.
Everything replaced from crt0 up.

We can do it  ;D
Title: Re: A microcontroller programming language
Post by: DiTBho on December 31, 2022, 02:23:34 am
Is this is going off topic. Xeon processors, MIPS, Athlon, power10?

We are talking about tr-mem, which I personally need as a "language feature", indeed I added specific support and basic-rule-set in my-c.

Unfortunately tr-mem seems very "implementation dependent", that's why we ended considering different platforms, from the success of IBM at implementing and supporting tr-mem in their POWER9-10 to the failure of Intel in making weak solutions that nobody is willing to give support, plus an experimental attempt made by a company that sent me a MIPS5++ prototype where tr-mem looks the simplest solution ever seen.

Title: Re: A microcontroller programming language
Post by: DiTBho on December 31, 2022, 02:38:11 am
Replace it with my_c.lib   :-DD   :o :o :o

LOL  :-DD

funny, just... well, my-c doesn't have any machine layer for m68k yet, only mips5++ is supported.

(one thing I learned...
adding machine-layer isn't as easy as expected, unless you leverage clang for your language... my-c was written from scratch from the lexer up, which is good in some respects and also a BIG mistake considering the effort required to support other architectures
... d'oh)

So, it will be libc-core-v1.c for m68k-elf-none-gcc-v9.3.0  ;D

Consider that my-c is 95% compatible with a subset of C89 (MisraC95&DO178B), so ... I will be probably able to re-use most of the code

(  :phew: )
Title: Re: A microcontroller programming language
Post by: SiliconWizard on December 31, 2022, 02:46:30 am
replacing the standard C library with something more suitable for my needs

That's my plan for 2023 for m68k.
Everything replaced from crt0 up.

We can do it  ;D

Of course.
Does it have to comply with the standard though? That would be quite limiting.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 31, 2022, 04:00:49 am
I'm thinking a microcontroller has less CPU grunt but is augmented by a few onboard PWMs, extra timers, voltage references, comparators, ADC and DAC ability, USARTs, onboard USB, I2C, SPI, WDT, writable permanent memory, sleep modes and commonly low powered so it can be run off battery power. There may be some offloading of functions such as DMA and gated timers. And commonly just 1 CPU.
Yes.

How do you then optimize for different architectures - AVR with multiple registers but transfers to memory are load and store - PIC with one working register but instructions like INCFSZ (increment memory and skip if zero) that doesn't touch the register at all.
That's exactly why I said I would focuse on the multiple general purpose register architectures: AVR on 8-bit, ARM/RISC-V/x86-64 on 32- and 64-bit architectures.

You really need a different approach on the backend to target the accumulator-style 8-bitters like 8051 etc.; just compare SDCC (https://sdcc.sourceforge.net/), GCC, and Clang targets for C.

Transactional memory and atomic memory access is discussed here, because it is a very similar problem on larger, multi-core processors.  There are three basic ways of implementing atomic memory access:Higher-level primitives like mutexes, semaphores, condition variables are implemented in terms of the above atomic operations.  However, there are also a large number of lockless data structures that rely on atomic access, and they tend to differ a bit between CAS and LL/SC.
I'm sure you can imagine how problematic it is to expose the low-level atomic operations in a low-level programming language: if you expose just CAS, the code will be suboptimal on LL/SC architectures, and vice versa.  If you provide both, how is the developer going to decide which one to use?

As you can see, you really need to understand the hardware differences to be able to determine what kind of language approach is suitable on use with a MCU.

A good example of this is the subset of C++ used in freestanding C++ environments.  Exceptions are usually not supported at all, because they're "designed" to be implemented via stack unwinding, and that's just too much work on microcontrollers for so little return.

Perhaps the best course is to wait for Sherlock to produce the language, and see if what benefits (if any) it brings. And with something to work with, then make comment to improve (if feasible).
My beef is that they're taking an already well-trodden path that has not thus far yielded anything practically useful.  Why repeat that?  I'm not saying this because I hope to benefit from their work, I am saying this because I've seen too many people get frustrated and fail, and I don't want Sherlock to do so too.

Why not instead try and approach it with the intent of producing something practically useful, with eyes and mind open to what it will end up looking like, and leverage all the massive amounts of practical knowledge we have on compilers and languages and microcontrollers?

And why go for just another object-oriented imperative language, when there is good reason to expect better returns from say an event-based language – simply because there are decades of research and a lot of money behind existing object-oriented imperative languages like Rust, but very few people considering event-based programming languages, even though a lot of code we write for microcontrollers revolves around dealing with events?

I for one am not willing to just stand by and look silently when someone sets themselves up for a failure.  I will pipe up, even if they themselves would prefer me to stay quiet.  It's a personality flaw.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 31, 2022, 04:40:34 am
replacing the standard C library with something more suitable for my needs
That's my plan for 2023 for m68k.
Everything replaced from crt0 up.
It is less work than one might expect, actually, at least for hosted environments ("userspace" code, running under a kernel providing I/O).
The trick is to start with a static inline extended assembly syscall wrappers, and implement the exported functions on top of those.
If you keep to the kernel interfaces, as opposed to standard C library interfaces, you'll also see significant code size savings.

Environment variables are nearly trivial if used in read-only manner, values deleted, or values replaced with equal length or shorter values.  It is modification and appending (setenv(), putenv() in C) that get hairy, especially if you want to avoid wasting RAM.

How you end up implementing formatted input and output – printf(), scanf() in C – is a more complicated question, but even there, it is more work to try and find out what kind of interfaces one really wants (again, if RAM and CPU use is to be minimized), than implementing them.

This is also why I believe a small but fundamental change in C would solve the most common bugs: replacing pointers with arrays as the base primitive, so that instead of working with pointers, one would work with contiguous memory ranges.  At the machine code level, nothing much would change; it would affect the function signatures and overall code at the conceptual level.  Things like string literals and other arrays wouldn't decay to a pointer, and instead would conceptually be a pointer, length tuple.
It is just that current C compilers are so darned complex already, that I don't know if that kind of fundamental change can be done as a modification, or whether one should start (a new front-end, for GCC or Clang, instead of modifying the existing C front-end) from scratch.
I do know that the GCC Fortran front-end heavily uses a similar concept for arrays and slices –– it is why certain linear algebra code written in Fortran runs faster than equivalent C code, both compiled using GCC.
Title: Re: A microcontroller programming language
Post by: brucehoult on December 31, 2022, 08:41:57 am
Transactional memory and atomic memory access is discussed here, because it is a very similar problem on larger, multi-core processors.  There are three basic ways of implementing atomic memory access:
  • Transactional memory.  Writes to transactional memory are grouped, and only actually committed to the underlying memory (atomically) if there are no conflicts.
  • CAS: Compare-exchange.  The basic operation is a conditional compare-exchange, that atomically replaces the memory contents if the previous contents match a given value.
  • LL/SC: Load-linked, store-conditional.  The basic operation is a linked load, which basically "claims" ownership of that memory address.  Within a few instructions (often with restrictions like no jumps), a corresponding linked store to that address will succeed if the contents were not modified in between.
Higher-level primitives like mutexes, semaphores, condition variables are implemented in terms of the above atomic operations.  However, there are also a large number of lockless data structures that rely on atomic access, and they tend to differ a bit between CAS and LL/SC.
I'm sure you can imagine how problematic it is to expose the low-level atomic operations in a low-level programming language: if you expose just CAS, the code will be suboptimal on LL/SC architectures, and vice versa.  If you provide both, how is the developer going to decide which one to use?

You've missed a class: Atomic Memory Operations, as seen in a number of current ISAs.

Semantically, this involves fetching a value from RAM (or a peripheral register), performing an arithmetic operation such as add, sub, and, or, xor, min, max, swap between the fetched value and a value in a register, and writing the result back to RAM or IO register. The result of the instruction is the original value of the memory location. This is all guaranteed to be atomic, and can not fail.

RISC-V has such instructions, as does ARMv8.1.

This seems on the face expensive and not scalable, but Berkeley's "TileLink" bus (enhanced and used by SiFive) enables the memory address, a constant, and the operation to be performed to be sent out to a suitably intelligent node in the memory system -- often the L2 cache shared by a number of CPU cores, but it might even be an IO device -- and the ALU operation is performed right where the data is stored, and only the result sent back to the CPU (and only if it's not going to be ignored by being written to register X0).

This scales extremely well.

Arm added a similar capability in AMBA 5 CHI (Coherent Hub Interface)
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 31, 2022, 11:00:51 am
Transactional memory and atomic memory access is discussed here, because it is a very similar problem on larger, multi-core processors.  There are three basic ways of implementing atomic memory access:
  • Transactional memory.  Writes to transactional memory are grouped, and only actually committed to the underlying memory (atomically) if there are no conflicts.
  • CAS: Compare-exchange.  The basic operation is a conditional compare-exchange, that atomically replaces the memory contents if the previous contents match a given value.
  • LL/SC: Load-linked, store-conditional.  The basic operation is a linked load, which basically "claims" ownership of that memory address.  Within a few instructions (often with restrictions like no jumps), a corresponding linked store to that address will succeed if the contents were not modified in between.
Higher-level primitives like mutexes, semaphores, condition variables are implemented in terms of the above atomic operations.  However, there are also a large number of lockless data structures that rely on atomic access, and they tend to differ a bit between CAS and LL/SC.
I'm sure you can imagine how problematic it is to expose the low-level atomic operations in a low-level programming language: if you expose just CAS, the code will be suboptimal on LL/SC architectures, and vice versa.  If you provide both, how is the developer going to decide which one to use?

You've missed a class: Atomic Memory Operations, as seen in a number of current ISAs.
You're right, as usual. :)

I did not consider them a separate class, because GCC, Clang, and Intel Compiler Collection expose them (implemented using the hardware bus lock (https://www.felixcloutier.com/x86/lock) prefix for select instructions – CAS as LOCK CMPXCHG etc.) via atomic built-ins (https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html), and I do use them when implementing lockless data structures on x86-64; but it is CAS, LL/SC, or transactional memory I'd need to implement the higher-level primitives like mutexes.

This seems on the face expensive and not scalable, but Berkeley's "TileLink" bus (enhanced and used by SiFive) enables the memory address, a constant, and the operation to be performed to be sent out to a suitably intelligent node in the memory system -- often the L2 cache shared by a number of CPU cores, but it might even be an IO device -- and the ALU operation is performed right where the data is stored, and only the result sent back to the CPU (and only if it's not going to be ignored by being written to register X0).
Extremely interesting!

(It also indicates that my expectation/hope of such tiny ALUs becoming more widely used may not be as crazy as it sounds.  An ALU micro-core between a DMA engine and I/O pins only needs a few instructions and a couple of registers to be able to do I2C, SPI, UART, PWM, PDM, and multi-bit 8080-style parallel buses.  The one Raspberry Pi folks created is too limited (no ALU), and the TI Sitara PRUs are too complex (full cores, with full access to shared RAM).  Something in between would be truly useful.)
Title: Re: A microcontroller programming language
Post by: tggzzz on December 31, 2022, 11:22:50 am
There are 8-bit architectures that have very few registers and were designed to pass function arguments on the stack, but they're quite difficult to optimize code for.  Most hardware today, from AVRs to ARMs to RISCV, have many general-purpose registers, so targeting those makes more sense to me anyway.
Not always true...
Well, I do think 12 general purpose registers (xCORE-200 XS2 ISA (PDF) (https://www.xmos.ai/download/xCORE-200:-The-XMOS-XS2-Architecture-%28ISA%29%281.1%29.pdf)) is plenty! ^-^

You are correct, of course :)

I'll claim, without proof (  ::) ), that another advantage of registers is that it increases instruction encoding density. That can be important for increasing speed and reducing program memory usage.

Nonetheless, matching memory speed to core speed helps a lot w.r.t. hardware complexity and MIPS/Watt.

Quote
But, sure, there are exceptions to any rule of thumb.  XCore is definitely one.

Yup, and it is an excellent example of what can be achieved when people decide they aren't interested in making incremental changes to existing cancerous growths, or archeological layered ruins if you prefer that analogy :)

Quote
That was demonstrated very clearly with the Sun UltrasSPARC Niagara T series processors, which had up to 128 "cores" (and used them effectively!) in 2005-2011, when x86-64 has <8 cores.
Yep.  I personally like the idea of asymmetric multiprocessing a lot, and would love to play with such hardware.  Alas, the only ones I have are mixed Cortex-A cores (ARM big.LITTLE (https://en.wikipedia.org/wiki/ARM_big.LITTLE)).

I'm not a fan of asymmetric processing for applications which aren't nailed down when the device ships. The asymmetry introduces a nasty question of whether a particular operation should be done here or there. That was a downfall of the CELL(?) architecture with one BIG and 8(?) LITTLEs a decade ago.

Where the only operations that will be in LITTLE are fixed at design time, that disadvantage is not a major problem.

Quote
We can see from the increasing complexity of peripherals on even cheap ARM Cortex-M microcontrollers, and things like Raspberry Pi Pico/RP2040 (and of course TI AM335x with its PRUs) that we're slowly going that way anyway.

Personally, I'd love to have tiny programmable cores with just a basic ALU – addition, subtraction, comparison, and bit operations, with just a couple of run-time registers – and access to reading and writing from the main memory, and the ability to raise an interrupt in the main core.  Heck, all the buses I use could be implemented with one.

When I was young, creating processors/ISAs was just about possible using bitslices like the 2900 family.

Now it is easy using FPGAs. Go do it :)

Quote
I looked at XEF216-512-TQ128-C20A (23.16€ in singles at Digikey, in stock), and the problem I have with it is that it is too powerful! :o

They have smaller ones in the family :)

Quote
I fully understand why XMOS developed xC, a variant of C with additions for tile computing; something like this would be perfect for developing a new event-based programming language, since it already has the hardware support for it.

Neither hardware nor software on their own are sufficient - as demonstrated many times over the decades.

The xC+xCORE software+hardware ecosystem is the key achievement. XMOS definitely got that right, even if people's imagination is too limited to see it  ::)

Quote
For now, however, I have my own sights set much lower: replacing the standard C library with something more suitable for my needs, to be used in a mixed freestanding C/C++ environment.  This is well defined in the standards, and there are more than one compiler I can use it with (specifically, GCC and Clang on different architectures, including 8-bit (AVR), 32-bit (ARM, x86), and 64-bit (x86-64, Aarch64/ARM64) ones).

I presume you are familiar with the Mill architecture. That's the only other non-incremental architecture worth looking at. The architects have a remarkable knowledge of what has worked well in the non-C non-UNIX world (yes, it exists) and have developed something that has DSP-like instruction parallelism and solid security while still working very well with C+UNIX and general-purpose computations.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 31, 2022, 11:29:46 am
...
As you can see, you really need to understand the hardware differences to be able to determine what kind of language approach is suitable on use with a MCU.

Precisely.

Unfortunately that requires a "renaissance man", and they are dying out, and growing new ones is almost impossible due to the cliff of existing details they have to absorb. The cliff was much easier when it was being created, step by step :)


Quote
...
And why go for just another object-oriented imperative language, when there is good reason to expect better returns from say an event-based language – simply because there are decades of research and a lot of money behind existing object-oriented imperative languages like Rust, but very few people considering event-based programming languages, even though a lot of code we write for microcontrollers revolves around dealing with events?

I for one am not willing to just stand by and look silently when someone sets themselves up for a failure.  I will pipe up, even if they themselves would prefer me to stay quiet.  It's a personality flaw.

Precisely.

Computer scientists tend to ignore the embedded world as being mere hardware. Damn their eyes!

And I have that personality flaw too :(
Title: Re: A microcontroller programming language
Post by: tggzzz on December 31, 2022, 11:33:09 am
This seems on the face expensive and not scalable, but Berkeley's "TileLink" bus (enhanced and used by SiFive) enables the memory address, a constant, and the operation to be performed to be sent out to a suitably intelligent node in the memory system -- often the L2 cache shared by a number of CPU cores, but it might even be an IO device -- and the ALU operation is performed right where the data is stored, and only the result sent back to the CPU (and only if it's not going to be ignored by being written to register X0).

This scales extremely well.

Arm added a similar capability in AMBA 5 CHI (Coherent Hub Interface)

I don't know the TileLink bus, but at that level it sounds similar to the hypertransport comms in AMD's Athlon/Opteron/etc. That works nicely up to a certain point and the cache coherence traffic and latency becomes a limiting factor.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on December 31, 2022, 01:47:08 pm
I'm not a fan of asymmetric processing for applications which aren't nailed down when the device ships. The asymmetry introduces a nasty question of whether a particular operation should be done here or there. That was a downfall of the CELL(?) architecture with one BIG and 8(?) LITTLEs a decade ago.
Ah, yes: IBM Cell (https://en.wikipedia.org/wiki/Cell_(microprocessor)) architecture, used on the Sony Playstation 3.

Where the only operations that will be in LITTLE are fixed at design time, that disadvantage is not a major problem.
I'm specifically talking about small "accelerators" with tiny instruction sets, basically a simple ALU (I don't even need multiplication or division myself!), attached to e.g. a DMA engine, GPIO ports, that sort of a thing; not full asymmetric cores, and not with full or random memory access.

Stuff like CRC or hash calculation while doing memory-to-memory DMA, peripheral bus implementations, PWM, PDM, ADC with averaging and min-max recording, even wide LUT.



To circle back to the original topic, it would be very nice if we had a programming language that could embed such "miniprograms" better than what we have right now.  I don't know how many of you are aware, but GPU pixel and vertex shaders are written in a C or C++ -like languages, for example OpenGL and OpenGL ES GLSL (https://en.wikipedia.org/wiki/OpenGL_Shading_Language) supplied as strings in source format, and compiled by the graphics drivers for its own hardware.

Perhaps some kind of augmented string format, or "#include" on steroids; perhaps a source file format where initial lines declared properties (accessible as compile-time constants), with the rest of the file available as an augmented string object?  Or processable with an external compiler program, with the result provided as an augmented binary blob?  The compile-time constant properties of that object are useful, if one needs to e.g. describe the resource use or expectations of that miniprogram.

In Linux systems programming, I occasionally use the seccomp BPF (https://www.kernel.org/doc/html/latest/userspace-api/seccomp_filter.html) filters, to limit a process to a subset of (explicitly allowed) system calls.  It mitigates the attack and bug surface when running e.g. dynamically compiled code  –– consider something that evaluates an user-defined expression a few hundred million times.  BPF itself is a mini-language (https://www.kernel.org/doc/html/latest/bpf/), with a small binary instruction set (https://www.kernel.org/doc/html/latest/bpf/instruction-set.html).  These are "installed" for a thread or process, and run by the kernel before internal syscall dispatch.  Currently, one writes such programs by populating an array using lots of preprocessor macros, which is otherwise OK, but calculating jumps is annoying:
Code: [Select]
static const struct sock_filter  strict_filter[] = {
    BPF_STMT(BPF_LD | BPF_W | BPF_ABS, (offsetof (struct seccomp_data, nr))),

    BPF_JUMP(BPF_JMP | BPF_JEQ, SYS_rt_sigreturn, 5, 0),  // Jump to RET_ALLOW if match
    BPF_JUMP(BPF_JMP | BPF_JEQ, SYS_read,         4, 0),  // Jump to RET_ALLOW if match
    BPF_JUMP(BPF_JMP | BPF_JEQ, SYS_write,        3, 0),  // Jump to RET_ALLOW if match
    BPF_JUMP(BPF_JMP | BPF_JEQ, SYS_exit,         2, 0),  // Jump to RET_ALLOW if match
    BPF_JUMP(BPF_JMP | BPF_JEQ, SYS_exit_group,   1, 0),  // Jump to RET_ALLOW if match

    BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_KILL),
    BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_ALLOW)
};

static const struct sock_fprog  strict = {
    .len = (unsigned short)( sizeof strict_filter / sizeof strict_filter[0] ),
    .filter = (struct sock_filter *)strict_filter
};

The RP2040 PIO and TI Sitara PRU code are examples of similar "miniprograms" in microcontroller environments.

Exactly how the "sub-language" embedding would work, I'm not sure –– I'd like the "give me this file as an augmented string object at compile time" and "run this file through an external compiler and give me the result as an augmented binary blob" ––; but looking at existing code and build machineries using such miniprograms/sub-languages might yield important ideas that reduces the barrier/learning-curve for using such sub-languages more often.

The way GCC, Clang, and Intel CC support GCC-style extended asm (https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html) should not be overlooked, either.  This is an extremely powerful macro assembler, that lets one write assembly where the C compiler chooses the exact registers used for the assembly snippet, unlike e.g. externally included assembly source files.  (With inlined functions whose bodies are basically just an extended asm statement, it means the compiler can choose which registers it uses in the inlined copy.)
The syntax for the constraints is absolutely horrible, though; otherwise it is pretty nice.
Title: Re: A microcontroller programming language
Post by: tggzzz on December 31, 2022, 03:07:50 pm
I'm not a fan of asymmetric processing for applications which aren't nailed down when the device ships. The asymmetry introduces a nasty question of whether a particular operation should be done here or there. That was a downfall of the CELL(?) architecture with one BIG and 8(?) LITTLEs a decade ago.
Ah, yes: IBM Cell (https://en.wikipedia.org/wiki/Cell_(microprocessor)) architecture, used on the Sony Playstation 3.

Where the only operations that will be in LITTLE are fixed at design time, that disadvantage is not a major problem.
I'm specifically talking about small "accelerators" with tiny instruction sets, basically a simple ALU (I don't even need multiplication or division myself!), attached to e.g. a DMA engine, GPIO ports, that sort of a thing; not full asymmetric cores, and not with full or random memory access.

Stuff like CRC or hash calculation while doing memory-to-memory DMA, peripheral bus implementations, PWM, PDM, ADC with averaging and min-max recording, even wide LUT.

Sounds a bit like the IBM360 channel processors, or network i/o processors.

By and large trying to offload i/o onto a separate processor only works in some cases; in many (most?) it is a problem.

The canonical example is offloading network protocols into a separate processor. Two intractable problems are:

If all you are attempting is to limit the unpredictability of i/o operations' timing, then a separate i/o processor can work well. And with that we're back to xCORE!
Title: Re: A microcontroller programming language
Post by: brucehoult on January 01, 2023, 12:53:12 am
This seems on the face expensive and not scalable, but Berkeley's "TileLink" bus (enhanced and used by SiFive) enables the memory address, a constant, and the operation to be performed to be sent out to a suitably intelligent node in the memory system -- often the L2 cache shared by a number of CPU cores, but it might even be an IO device -- and the ALU operation is performed right where the data is stored, and only the result sent back to the CPU (and only if it's not going to be ignored by being written to register X0).

This scales extremely well.

Arm added a similar capability in AMBA 5 CHI (Coherent Hub Interface)

I don't know the TileLink bus, but at that level it sounds similar to the hypertransport comms in AMD's Athlon/Opteron/etc. That works nicely up to a certain point and the cache coherence traffic and latency becomes a limiting factor.

It's not cache-coherence. The whole point is to be able to avoid moving cache lines around.

Everything works up to a certain point. The question to be answered is whether that point is 4 CPUs, 64 CPUs, or 1024+ CPUs.

x86's (and SPARC's) TSO memory model is also a limiting factor for those systems. Arm and RISC-V have better memory model, certainly by the time the CPU count gets into three figures.

Note that there are already RISC-V chips in production with more than 1000 cores e.g. ET-SoC-1 with 1088 small in-order cores with vector units, plus 4 big OoO cores.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on January 01, 2023, 04:38:14 am
By and large trying to offload i/o onto a separate processor only works in some cases; in many (most?) it is a problem.
[...]
If all you are attempting is to limit the unpredictability of i/o operations' timing, then a separate i/o processor can work well. And with that we're back to xCORE!
No, actually neither.  And I'm very often fighting buffer bloat, so I'm looking to ways of avoid that, too.

The idea with the small ALUs is not to have more buffers, but to do whatever small arithmetic and bit operations are necessary when the data is accessed anyway, and use as few buffers as is possible.  Zero-copy operation, really, is what I'm after.

The common Ethernet interface I see on MCUs (like NXP i.MX RT1062 and STM32F417) is a set of registers and packet data fed via FIFO, usually using a DMA channel.  BPF was designed as a micro-language to examine the packets as they were received, to early discard and route them.  (Even on servers, I never liked Ethernet offload, because it does add significant buffer blots; I'm not talking about that kind of offloading.)

For Ethernet interface, internal "routing" –– to the code that is interested in specific packets –– can be done whenever the IP packet headers are read; there is no need to wait for the data to complete before doing that.  One can even mark the incoming packet buffer to be immediately reused, if the packet is not something that needs handling.  A send operation can be completely asynchronous, just setting up the DMA descriptor for the packet, with a suitable event generated when the packet has been transmitted (UDP or raw Ethernet frame) or a TCP ACK received.
Title: Re: A microcontroller programming language
Post by: Siwastaja on January 01, 2023, 07:48:47 am
The idea with the small ALUs is not to have more buffers, but to do whatever small arithmetic and bit operations are necessary when the data is accessed anyway, and use as few buffers as is possible.  Zero-copy operation, really, is what I'm after.

Exactly. A simple example:

Many UART or SPI based protocols add a CRC16 for data integrity verification. Yet, microcontroller peripherals usually cannot calculate CRCs. So all practical code implementations either prepare the complete message in the buffer and calculate CRC in one go - allowing DMA to be used, but timing is awkward as the CRC calculation over the whole message takes so long. Another option is to calculate CRC word per word during RX / TX - which is not very high cost operation if you process the data in interrupt handler word per word anyway. But then you can't use DMA.

Higher end STM32 SPI peripherals include hardware CRC for this reason, but being fixed in silicon, it can only cover some cases (there is no standardized "CRC over SPI").

What you describe would do exactly that, but in much more flexible way. What's even better, such simple core could bit-bang the protocol itself, so the microcontroller would not need a separate UART / I2C / SPI peripherals at all.

Two intractable problems are:
  • the network processor is slower...
  • ...waiting for the (slow) network processor...

Really, the one problem is slow speed. The solution is to make them fast. There is no reason why the simple cores could not run at comparable speed to the main CPU. In order not to make them complicated with pipelining, branch prediction etc., it means giving up some of their capabilities, but that's OK. Even if you have just basic 32-bit add and bitwise operations plus branch instructions, running at say 200MHz while the more complex main CPU runs heavily pipelined at 400MHz, that's OK. If you look at current high-end microcontroller offerings, their IO and most peripherals already run at Fcpu/2 or lower. One could just replace the hardwired peripherals with tiny, simple CPUs.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on January 01, 2023, 10:36:58 am
Even if you have just basic 32-bit add and bitwise operations plus branch instructions, running at say 200MHz while the more complex main CPU runs heavily pipelined at 400MHz, that's OK. If you look at current high-end microcontroller offerings, their IO and most peripherals already run at Fcpu/2 or lower. One could just replace the hardwired peripherals with tiny, simple CPUs.
I agree.  Furthermore, these tiny cores – really, just ALUs – don't even need arbitrary memory address access, only a couple of accumulator registers (although a dedicated read-only lookup access would be nice), and very limited loop/jump/conditional capabilities.  They're truly simple.

Something like 100 MHz would suffice for my own peripheral bus needs quite nicely (even when the MCU core itself runs at say 400-1000 MHz).

Still, it is the programming of such tiny limited cores in a sub-programming-language within larger applications that currently keeps them in "niche" uses, I believe.
I know tggzzz and others somewhat disagree, but I've found them (as I've described, from BPF to shaders) extremely powerful, and only difficult to "teach" others to use efficiently and effectively.  (Not the programming part itself; but the design and integration part.)

In my opinion, it is exactly the development of programming paradigms –– especially event-based programming languages –– that is keeping us software folks back, when compared to the progress made on the hardware side in the last three decades or so.  I mean, compilers have become cleverer and better, and we have more choices when it comes to programming languages and especially libraries, but real forward development is rare!

It is also why I do not want to discourage people like OP who want to design new languages; I only want them to consider approaches that haven't already been trodden by scores of well-resourced developers without real significant results; to see approaches that my and others' experience indicates might yield much better results, something new.

(Okay, I am actually well aware that the OP does not read any of my posts, so I'm not writing these posts to OP.  I'm writing these to the other people who are mulling new microcontroller programming language development now or in the future, in the hopes that the concepts I'm pushing leads them to discover and develop something truly better than the imperative languages we have now, without complicated layers of abstractions and runtimes that require more and more computing resources like we see on server and desktop architectures.)
Title: Re: A microcontroller programming language
Post by: Siwastaja on January 01, 2023, 11:00:31 am
these tiny cores
...
Still, it is the programming of such tiny limited cores in a sub-programming-language within larger applications that currently keeps them in "niche" uses, I believe.

These tiny cores could be register-only machines with say 16-32 general purpose registers* (to be able to hold at least some state) and no RAM/stack. Typical program would be tens, max hundreds of LoC. The instruction set could be easy enough that everyone would just program them in assembly. With no memory addressing or other such complicated stuff to parse, the assembler itself could be <1K of code (direct, simple parsing of mnemonics to opcodes), so maybe one could even write the assembly code within the main MCU code and have main CPU assemble it at runtime. (Maybe the machine code would be doable compile-time with some clever C preprocessor stuff, even.)

EDIT: *) with a mechanism to configure the reset values of the registers, for constants, think about CRC polynomial for example.
Title: Re: A microcontroller programming language
Post by: Nominal Animal on January 01, 2023, 11:09:31 am
Consider the common Unix command-line sort utility.

Regardless of what programming language you use, you typically implement it as reading the input into an array of lines, then sort that array, and finally output the sorted array.

This is very suboptimal use of computing resources.

Instead, one should use a self-organizing data structure, for example a binary heap, so that each line read gets sorted and "inserted" into the correct position.  This way, the data is sorted immediately when it is received, and you have best cache locality given the situation.  You also use the CPU time that otherwise would be mostly wasted in waiting for the I/O to complete, for computation on data that has already been touched recently.

Because the I/O bandwidth is the bottleneck here –– sorting/insertion does not require much computation, just comparisons ––, in real world time, this approach can start emitting the "sorted" data immediately after the last line of input has been read, when the traditional approach is just ready to start sorting the data.

Overall, self-organizing data structure insertion operations might use fractionally more CPU time than the traditional approach, because we have some very clever offline array sort algorithms.  It depends on the instruction set, hardware architecture, and especially the cache architecture, which one actually consumes fewer resources overall.

It is this kind of algorithmic approach that will lead to true progress in software development.  Compiler optimization is just details.  Problem is, how do we make it easier for humans to implement algorithms that yield better solutions, when the imperative approach they find intuitive favors the less efficient algorithms?

Let's consider my suggestion of focusing on event-based programming, and what a sort implemented using one would be like.

For one, we'll need the input to produce lines, preferably with each line a separate event we can act on.  Then, reading the input is a matter of setting up the inputs as input line event generators, and the event handler inserts the lines to a suitable data structure, for example an extensible binary min/max-heap stored in an array.  The sort key and direction rules would be applied at that insertion phase.  When the input has been completed – an end-of-input event? – we switch to the data structure, emptying it line-by-line in order to the output.  Done.

It is examples like this, and stuff like button/encoder-controlled menus on gadgets, finite state machines, and other very event-oriented stuff, that makes me think event-based programming languages might give us a leap forwards.  As I mentioned before, even server-side software tends to be very event-oriented, and their operation easier to describe in event terms – request, response especially – than in imperative terms.

Yet, I'm not particularly enamored by event-based programming.  For things like the ALU microcores mentioned in previous messages – including BPF, pixel and vertex shader programming –, I definitely prefer imperative programming.
It interests me because I see it as a possible way out of the decades-long stuck phase in software development!  (Okay, object oriented programming has made user interface programming much, much easier and arguably more effective than before, but it is just about the only field I've seen truly progress forward; and yet, even it hasn't made any big difference in software reliability, effectiveness –– and definitely has worsened memory hogging.)
In other words, I see us software folks stuck in abstractions and paradigms that haven't really evolved much in the last three decades or more, and am trying to get others interested in the largely untested approaches, because the imperative-object-oriented approaches don't seem to be able to produce anything clearly better, even with decades of person-hours of development work in them.
Title: Re: A microcontroller programming language
Post by: tggzzz on January 01, 2023, 11:38:12 am
...
Something like 100 MHz would suffice for my own peripheral bus needs quite nicely (even when the MCU core itself runs at say 400-1000 MHz).

Still, it is the programming of such tiny limited cores in a sub-programming-language within larger applications that currently keeps them in "niche" uses, I believe.
I know tggzzz and others somewhat disagree, but I've found them (as I've described, from BPF to shaders) extremely powerful, and only difficult to "teach" others to use efficiently and effectively.  (Not the programming part itself; but the design and integration part.)

I'm not sure we do disagree there. Domain Specific Languages can be an excellent architectural choice, but usually aren't.

The "usually bad" examples are typically from softies that yearn to invent their own language (the hardware equivalent is inventing their own processor).

So what might I think are (potentially) good examples?

Having said that, teaching effective use of such languages will remain a problem. It may be interesting to consider why it is difficult.
Is it more difficult that teaching standard programming paradigms? I tend to doubt that, based on the messes people produce in standard paradigms!
Perhaps people's minds are crippled by learning only the standard paradigms? (cf Dijkstra on COBOL)

Quote
In my opinion, it is exactly the development of programming paradigms –– especially event-based programming languages –– that is keeping us software folks back, when compared to the progress made on the hardware side in the last three decades or so.  I mean, compilers have become cleverer and better, and we have more choices when it comes to programming languages and especially libraries, but real forward development is rare!

Oh yes! Precisely!

Quote
It is also why I do not want to discourage people like OP who want to design new languages; I only want them to consider approaches that haven't already been trodden by scores of well-resourced developers without real significant results; to see approaches that my and others' experience indicates might yield much better results, something new.

(Okay, I am actually well aware that the OP does not read any of my posts, so I'm not writing these posts to OP.  I'm writing these to the other people who are mulling new microcontroller programming language development now or in the future, in the hopes that the concepts I'm pushing leads them to discover and develop something truly better than the imperative languages we have now, without complicated layers of abstractions and runtimes that require more and more computing resources like we see on server and desktop architectures.)

Oh yes! Precisely!
Title: Re: A microcontroller programming language
Post by: tggzzz on January 01, 2023, 11:58:03 am
This seems on the face expensive and not scalable, but Berkeley's "TileLink" bus (enhanced and used by SiFive) enables the memory address, a constant, and the operation to be performed to be sent out to a suitably intelligent node in the memory system -- often the L2 cache shared by a number of CPU cores, but it might even be an IO device -- and the ALU operation is performed right where the data is stored, and only the result sent back to the CPU (and only if it's not going to be ignored by being written to register X0).

This scales extremely well.

Arm added a similar capability in AMBA 5 CHI (Coherent Hub Interface)

I don't know the TileLink bus, but at that level it sounds similar to the hypertransport comms in AMD's Athlon/Opteron/etc. That works nicely up to a certain point and the cache coherence traffic and latency becomes a limiting factor.

It's not cache-coherence. The whole point is to be able to avoid moving cache lines around.

That's possible and desirable except where memory is shared between cores, e.g. buffers and synchronisation mechanisms.

Quote
Everything works up to a certain point. The question to be answered is whether that point is 4 CPUs, 64 CPUs, or 1024+ CPUs.

x86's (and SPARC's) TSO memory model is also a limiting factor for those systems. Arm and RISC-V have better memory model, certainly by the time the CPU count gets into three figures.

Note that there are already RISC-V chips in production with more than 1000 cores e.g. ET-SoC-1 with 1088 small in-order cores with vector units, plus 4 big OoO cores.

I can't comment on the relative quality of memory models, since it has been too long since I was interested in them.

There's a general problem of deciding the best granularity for parallel computation. Too fine a granularity and little context has to be shared but the synchronisation times will dominate the computation times. Too coarse, and synchronisation is less of a problem but communicating more context becomes an issue.

Where exactly a cliff lies will depend on both the low-level architecture and the application. Some applications are "embarassingly parallel", and for those the cliff will be more remote and/or gentle. Others will hit Amdahl's law sooner.

I am skeptical about putting vast numbers of general purpose cores on a single piece of silicon; on chip memory size and off-chip bandwidth+latency rapidly become limiting factors. For some niche applications that is not a limitation, of course.
Title: Re: A microcontroller programming language
Post by: tggzzz on January 01, 2023, 12:23:23 pm
Yet, I'm not particularly enamored by event-based programming.  For things like the ALU microcores mentioned in previous messages – including BPF, pixel and vertex shader programming –, I definitely prefer imperative programming.

Having the right granularity of event is the key; it should map directly onto the events in the application's definition and be understandable by someone not involved in the implementation.

There was a chip with, IIRC, 144 tiny cores+memory on it. I forget the manufacturer, but Greenpak/Greenpar springs to mind. I looked at it, but the only use case I could find used each core as a little more than a single logic gate - like a LUT in an FPGA.

Quote
It interests me because I see it as a possible way out of the decades-long stuck phase in software development!  (Okay, object oriented programming has made user interface programming much, much easier and arguably more effective than before, but it is just about the only field I've seen truly progress forward; and yet, even it hasn't made any big difference in software reliability, effectiveness –– and definitely has worsened memory hogging.)

OOP has had one major effect: it strongly encourages encapsulation at multiple levels from small objects to large sub-systems accessed only via interfaces. That reduces the cognitive load by reducing the number of things that have to be known before something is used as a black box component. That in turn enables larger applications to be composed by average programmers. And applications have become more complex until the problems of poor structuring limit them.

Summary: the same old problems of programmer's brainpower still apply, but at a larger/higher/more complex level.

And yes, OOP isn't necessary to enable large complex well structured applications, which can be done in C/Pascal/etc. But in practice it seems to be a precondition.

And yes, poor OOP technology leads to poor unreliable applications; think C++ (if you think C++ is the answer, then you need to revisit the question!)

Quote
In other words, I see us software folks stuck in abstractions and paradigms that haven't really evolved much in the last three decades or more, and am trying to get others interested in the largely untested approaches, because the imperative-object-oriented approaches don't seem to be able to produce anything clearly better, even with decades of person-hours of development work in them.

Yup. Me too :)
Title: Re: A microcontroller programming language
Post by: DiTBho on January 01, 2023, 12:52:44 pm
Quote
if you think C++ is the answer,
then you need to revisit the question!

Loooooooool, I want a t-shirt with this  :D :D :D
Title: Re: A microcontroller programming language
Post by: tggzzz on January 01, 2023, 12:56:13 pm
Quote
if you think C++ is the answer,
then you need to revisit the question!

Loooooooool, I want a t-shirt with this  :D :D :D

I decided that in the late 80s, and subsequent history has only reinforced my view!
Title: Re: A microcontroller programming language
Post by: Siwastaja on January 01, 2023, 01:43:05 pm
C++ is great, because C++ connects people: basically all experienced or knowledgeable people (people with a clue, or those who grok, so to speak) agree on if not hating, but at least not loving C++, no matter how much they disagree otherwise. It's truly a horrible and messy language.
Title: Re: A microcontroller programming language
Post by: tggzzz on January 01, 2023, 05:19:06 pm
C++ is great, because C++ connects people: basically all experienced or knowledgeable people (people with a clue, or those who grok, so to speak) agree on if not hating, but at least not loving C++, no matter how much they disagree otherwise. It's truly a horrible and messy language.

That is a superb example of "making a silk purse out of a sow's ear" (which has been done!). I'm far too old and grouchy to be able to be so imaginatively positive :(
Title: Re: A microcontroller programming language
Post by: SiliconWizard on January 01, 2023, 07:49:37 pm
C++ is great, because C++ connects people: basically all experienced or knowledgeable people (people with a clue, or those who grok, so to speak) agree on if not hating, but at least not loving C++, no matter how much they disagree otherwise. It's truly a horrible and messy language.

As I mentioned in other threads, one particularly entertaining activity (up to a point, don't abuse it) is to watch talks from C++ conferences such as CppCon.
Almost endless hours of fun.

And to be fair, although for slightly different reasons, Rust talks are often pretty entertaining too. Don't take my word for it, go watch a few!

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 01, 2023, 08:24:13 pm
Yes, all args are passed by ref. To achieve pass by value (that is, pass a ref to a temp copy) one just surrounds the arg in parens:

omfg. I am without words.

You just threw away a huge chunk of performance, as well as compatibility with all existing libraries and OS interfaces.

Ha! but not without expletives!

OK since you do want to discuss this with me, let me respond.

I'm referring to the general policy for arg passing, ultimately an implementation could do this in various ways (pass in a register for example). I'm not really referring to the implementation but to the semantics, I don't think I made that clear.

So "pass by reference" here is meant to indicate that the callee can change something in the datum that's been passed. That's the default convention, and if the the caller want's to prevent a callee from changing the value of an arg, they'd simply make that arg an expression with ( and ).

This leads to increased performance because unless the developer wants to pass by value, they can write the code and just let args pass by ref, meaning passing a bunch of structures, strings, arrays or combinations of these will entail nothing more than pushing a pointer onto the stack!

Ugh.

Code: [Select]
procedure foo(i : integer)
  var i : integer = 1;
  procedure bar(j : integer)
    i := i + 1;
    j := j + 1
  end
  bar(i);
  print(i)
end

What is printed?

Run it and it and see, then tell us the answer and then explain what the relevance is of asking this question? is it an attempt to make some "point"? well then state your case, state your proposition whatever it is.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 01, 2023, 08:28:09 pm
the author should really take a deep look into

Frankly, the author should really get a job. I mean programming real stuff, from home automation to avionics. Only then can he come back here and talk. Otherwise, sugar crap is and will be the only main core of everything.

Why do you want to discuss me rather than programming languages? I can talk here any time I choose, your approval is irrelevant, your criteria for my participation is irrelevant, please stick to the subject, be objective scientific, analytical rather than rude, insulting and emotional.

What has one's employment status got to do with the subject under discussion?
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 01, 2023, 08:35:13 pm
Why do you want to discuss me rather than programming languages?

Your idea have ZERO real life experience, too much arrogance.
Enjoy my ignore list.

More emotions, try answering my question before running away in future.

If you want to debate me on something then ask me, start a thread, state some position and if I disagree I'll respond and we can then debate it like adults, no emotions, just, logic, reason, ball's in your court.

This thread though is not about me, go and re-read the OP, see for yourself.


Title: Re: A microcontroller programming language
Post by: DiTBho on January 01, 2023, 08:38:27 pm
Why do you want to discuss me rather than programming languages?

Run it and it and see, then tell us the answer and then explain what the relevance is of asking this question? is it an attempt to make some "point"? well then state your case, state your proposition whatever it is.

The Dunning-Kruger effect occurs when a person's lack of knowledge and skills in a certain area
 (real life programming) cause them to overestimate their own competence.

Enjoy my ignore list.
Title: Re: A microcontroller programming language
Post by: tggzzz on January 01, 2023, 08:56:06 pm
C++ is great, because C++ connects people: basically all experienced or knowledgeable people (people with a clue, or those who grok, so to speak) agree on if not hating, but at least not loving C++, no matter how much they disagree otherwise. It's truly a horrible and messy language.

As I mentioned in other threads, one particularly entertaining activity (up to a point, don't abuse it) is to watch talks from C++ conferences such as CppCon.
Almost endless hours of fun.

And to be fair, although for slightly different reasons, Rust talks are often pretty entertaining too. Don't take my word for it, go watch a few!

I don't like 99.7% of videos: too few words/minute, and there's no way of determining within 30s that it will be worth my time. (Won't be worth my time is a trivially easy decision) Hence I get my C++ jollies from the C++ Frequently Questioned Answers[1] at https://yosefk.com/c++fqa/ Don't forget to click on any of the "FAQ:" links, since they link to the relevant FAQ.

Zealots claim that the FQA is no longer valid in some respects. I have no reason to doubt that, but there are plenty of others left untouched. I'm particularly fond of the const correctness (https://yosefk.com/c++fqa/const.html) section. As a taster I'll repeat "The pointer aliasing issue is one reason that the compiler can't really optimize code because it sees a pointer declared as const. And when it can figure out there are no aliases, it doesn't need your const declarations to help it."

If that is too detailed, then there's always https://yosefk.com/c++fqa/class.html#fqa-7.5 "How does C++ help with the tradeoff of safety vs. usability? ... Anyway, here's the real answer to the original question: C++ helps with the tradeoff of safety vs. usability by eliminating both."

[1] accurate, and sets the tone nicely
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 01, 2023, 08:58:51 pm
Why do you want to discuss me rather than programming languages?

Run it and it and see, then tell us the answer and then explain what the relevance is of asking this question? is it an attempt to make some "point"? well then state your case, state your proposition whatever it is.

The Dunning-Kruger effect occurs when a person's lack of knowledge and skills in a certain area
 (real life programming) cause them to overestimate their own competence.

Enjoy my ignore list.

This like many posts here, is fallacious, you cannot invalidate someone's argument on the basis of their job title, experience or educational qualifications, this is terrible reasoning, really poor reasoning.

"But Mr. Einstein, we can't accept this rather controversial pseudo scientific paper on clocks 'slowing down' and so on. I mean you're only a patent clerk as well for goodness sake, sorry we can't publish such ideas, not in our journal".

Title: Re: A microcontroller programming language
Post by: brucehoult on January 02, 2023, 09:02:51 am
Yes, all args are passed by ref. To achieve pass by value (that is, pass a ref to a temp copy) one just surrounds the arg in parens:

omfg. I am without words.

You just threw away a huge chunk of performance, as well as compatibility with all existing libraries and OS interfaces.

Ha! but not without expletives!

OK since you do want to discuss this with me, let me respond.

I'm referring to the general policy for arg passing, ultimately an implementation could do this in various ways (pass in a register for example). I'm not really referring to the implementation but to the semantics, I don't think I made that clear.

So "pass by reference" here is meant to indicate that the callee can change something in the datum that's been passed. That's the default convention, and if the the caller want's to prevent a callee from changing the value of an arg, they'd simply make that arg an expression with ( and ).

This leads to increased performance because unless the developer wants to pass by value, they can write the code and just let args pass by ref, meaning passing a bunch of structures, strings, arrays or combinations of these will entail nothing more than pushing a pointer onto the stack!

Ugh.

Code: [Select]
procedure foo(i : integer)
  var i : integer = 1;
  procedure bar(j : integer)
    i := i + 1;
    j := j + 1
  end
  bar(i);
  print(i)
end

What is printed?

Run it and it and see, then tell us the answer and then explain what the relevance is of asking this question? is it an attempt to make some "point"? well then state your case, state your proposition whatever it is.

I can not run it, sir.

This is my best guess of a possible syntax for YOUR proposed programming language, which you have not provided us with a compiler for.

My proposition is that your feature of everything is pass by reference not only sucks in general, but is especially nasty when combined with your feature of nested functions.
Title: Re: A microcontroller programming language
Post by: JPortici on January 02, 2023, 10:01:48 am
Frankly, the point bruce wanted to make was pretty obvious: anyone can write terrible code in any language, no matter the features that should supposedly make it better than *insert language name*. That snippet really hurt my brains and made me wish for a committee to produce a coding standard.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 02, 2023, 12:19:04 pm
Yes, all args are passed by ref. To achieve pass by value (that is, pass a ref to a temp copy) one just surrounds the arg in parens:

omfg. I am without words.

You just threw away a huge chunk of performance, as well as compatibility with all existing libraries and OS interfaces.

Ha! but not without expletives!

OK since you do want to discuss this with me, let me respond.

I'm referring to the general policy for arg passing, ultimately an implementation could do this in various ways (pass in a register for example). I'm not really referring to the implementation but to the semantics, I don't think I made that clear.

So "pass by reference" here is meant to indicate that the callee can change something in the datum that's been passed. That's the default convention, and if the the caller want's to prevent a callee from changing the value of an arg, they'd simply make that arg an expression with ( and ).

This leads to increased performance because unless the developer wants to pass by value, they can write the code and just let args pass by ref, meaning passing a bunch of structures, strings, arrays or combinations of these will entail nothing more than pushing a pointer onto the stack!

Ugh.

Code: [Select]
procedure foo(i : integer)
  var i : integer = 1;
  procedure bar(j : integer)
    i := i + 1;
    j := j + 1
  end
  bar(i);
  print(i)
end

What is printed?

Run it and it and see, then tell us the answer and then explain what the relevance is of asking this question? is it an attempt to make some "point"? well then state your case, state your proposition whatever it is.

I can not run it, sir.

This is my best guess of a possible syntax for YOUR proposed programming language, which you have not provided us with a compiler for.

My proposition is that your feature of everything is pass by reference not only sucks in general, but is especially nasty when combined with your feature of nested functions.

I'm not familiar with the technical terms "sucks" or "nasty" when describing programming languages.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 02, 2023, 09:20:08 pm
I have a feeling this language will suck more than what (C?) is going to replace  :o :o :o

If it's efficacy were dependent upon facile contributions like this post of yours, I dare say I'd agree with you.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 02, 2023, 09:28:31 pm
Note that, as we often discuss, this certainly has an impact on the intrisic qualities of a language, but little on its popularity. A number of languages that have become very popular were just a bunch of ad-hoc stuff stitched together. Sad but just how it goes.
PHP (https://en.wikipedia.org/wiki/PHP) is perhaps the best example of this.

Considering that, why would anyone start designing a language from the ground up, starting with syntax or grammar, instead of examining how expressions and algorithms would be best expressed in the new language?

How many time must I address this point?

First, language grammar's have properties, different grammars have different properties, a property I value greatly is an ability to easily extend the scope of the language by adding new keywords.

Second my posts have not been confined to matters of grammar, I've discussed a plethora of things so far, yet some here - like you - seem to want to do nothing but complain.

Third I will discuss what I want to discuss in any order I choose, if this is not to your liking then kindly leave the thread in peace.



Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 02, 2023, 09:44:31 pm
Here's a condensed list of important goals for a new language suitable for writing code for MCUs:

1. Extensible grammar (new keywords easily added over time, 100% backward compatibility guaranteed)
2. Elimination of pointer arithmetic.
3. Multidimensional arrays with variable upper/lower bounds.
4. An advanced preprocessor.
5. Support true 'offsets' in addition to conventional 'pointers'.
6. Supported nested procedures.
7. Support exceptions/conditions
8. Support 'builtin' and 'intrinsic' as fundamental language concepts.
9. Abstraction of interop - use a declarative means of interfacing to code written by other languages.
10. Support 'bit' as a data type.
11. Support flexible alignment, padding and ordering of structure/array members.
12. Support endianness operations for converting a contiguous memory block from big/little to little/big endian.
13. Support multiple 'heaps' and facilitate the freeing of all allocations within a heap by simply resetting the heap.
14. Innate support for coroutines.
15. Support decimal, octal, binary and hex numeric literals.
16. Support fixed point arithmetic in decimal/binary base.
17. Remain vigilante, aware of other old and new languages like Ada, Rust, Zig, Hare, Odin etc, where are these weak? where are they strong?

Is there anything fundamental missing from this list ? (I recognize that some want to talk about threading, synchronization and multiple cores, these are important but perhaps best set aside for now, to be revisited later).

It would be interesting to also examine various examples of C based solutions to certain MCU problems to see how a new language could improve the implementation of that code.

I'd like to also remind some here that all/any of these topics are up for discussion, I am not only concerned with grammar, if anyone wants to discuss any of these aspects then please be my guest, lets explore and move forward...



Title: Re: A microcontroller programming language
Post by: MIS42N on January 02, 2023, 10:09:40 pm

Code: [Select]
procedure foo(i : integer)
  var i : integer = 1;
  procedure bar(j : integer)
    i := i + 1;
    j := j + 1
  end
  bar(i);
  print(i)
end

What is printed?
I wondered if there was a sensible answer.

Hopefully the compiler would give a warning "Do you really want to create a local variable the same name as a passed parameter?" at line 2.

If that warning is ignored, then does procedure bar inherit the current instantiation of i from foo due to scoping?
if not, then compiler should warn "uninitiated variable i"
if so within bar, i := i + 1; will operate on the i declared in foo due to scoping: i then = 2
j := j + 1 will operate on the  i declared in foo as a passed variable by reference: i then = 3
Print 3

This seems logical to me. Makes what I regard as sensible interpretation of scoping. Is this a reasonable analysis? other interpretation?

Maybe a solution is to make the rules flexible [procedure bar(i) - don't say how i is passed and let the linker make that decision. Not the same as VB where the variable is mutable at run time, just flexible until link time]. If the procedure caller doesn't care what happens to the passed variables or requires their version to remain unchanged, can explicitly pass by value. If the procedure caller expects something to be modified and wants the result, explicitly pass by reference. Let the linker fix it up. This means the programmer has more flexibility, the procedure writer can determine if a particular method is required (causing a linker error if not adhered to) or let the linker fix it. And there is no ambiguity in the calling code.

Title: Re: A microcontroller programming language
Post by: brucehoult on January 02, 2023, 10:57:50 pm

Code: [Select]
procedure foo(i : integer)
  var i : integer = 1;
  procedure bar(j : integer)
    i := i + 1;
    j := j + 1
  end
  bar(i);
  print(i)
end

What is printed?
I wondered if there was a sensible answer.

Hopefully the compiler would give a warning "Do you really want to create a local variable the same name as a passed parameter?" at line 2.

If that warning is ignored, then does procedure bar inherit the current instantiation of i from foo due to scoping?
if not, then compiler should warn "uninitiated variable i"
if so within bar, i := i + 1; will operate on the i declared in foo due to scoping: i then = 2
j := j + 1 will operate on the  i declared in foo as a passed variable by reference: i then = 3
Print 3

This seems logical to me. Makes what I regard as sensible interpretation of scoping. Is this a reasonable analysis? other interpretation?

Maybe a solution is to make the rules flexible [procedure bar(i) - don't say how i is passed and let the linker make that decision. Not the same as VB where the variable is mutable at run time, just flexible until link time]. If the procedure caller doesn't care what happens to the passed variables or requires their version to remain unchanged, can explicitly pass by value. If the procedure caller expects something to be modified and wants the result, explicitly pass by reference. Let the linker fix it up. This means the programmer has more flexibility, the procedure writer can determine if a particular method is required (causing a linker error if not adhered to) or let the linker fix it. And there is no ambiguity in the calling code.

Very good except our friend stated "So 'pass by reference' here is meant to indicate that the callee can change something in the datum that's been passed", and in other places that it might not actually be by a pointer, but by being passed in a register and passed back. He's also indicated several times the he follows Ada in various things.

Ada has INOUT parameters, where it is *undefined* whether they are passed by copy/return or by pointer.

This makes a difference to code such as the above.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 02, 2023, 11:30:37 pm

Code: [Select]
procedure foo(i : integer)
  var i : integer = 1;
  procedure bar(j : integer)
    i := i + 1;
    j := j + 1
  end
  bar(i);
  print(i)
end

What is printed?
I wondered if there was a sensible answer.

Hopefully the compiler would give a warning "Do you really want to create a local variable the same name as a passed parameter?" at line 2.

If that warning is ignored, then does procedure bar inherit the current instantiation of i from foo due to scoping?
if not, then compiler should warn "uninitiated variable i"
if so within bar, i := i + 1; will operate on the i declared in foo due to scoping: i then = 2
j := j + 1 will operate on the  i declared in foo as a passed variable by reference: i then = 3
Print 3

This seems logical to me. Makes what I regard as sensible interpretation of scoping. Is this a reasonable analysis? other interpretation?

Maybe a solution is to make the rules flexible [procedure bar(i) - don't say how i is passed and let the linker make that decision. Not the same as VB where the variable is mutable at run time, just flexible until link time]. If the procedure caller doesn't care what happens to the passed variables or requires their version to remain unchanged, can explicitly pass by value. If the procedure caller expects something to be modified and wants the result, explicitly pass by reference. Let the linker fix it up. This means the programmer has more flexibility, the procedure writer can determine if a particular method is required (causing a linker error if not adhered to) or let the linker fix it. And there is no ambiguity in the calling code.

Very good except our friend stated "So 'pass by reference' here is meant to indicate that the callee can change something in the datum that's been passed", and in other places that it might not actually be by a pointer, but by being passed in a register and passed back. He's also indicated several times the he follows Ada in various things.

Ada has INOUT parameters, where it is *undefined* whether they are passed by copy/return or by pointer.

This makes a difference to code such as the above.

A similar discussion arises when we discuss stuff like:

Code: [Select]

for (I = 0; I < Funct(I); I++)
     {
     ;
     }

Is Funct(I) evaluated once, at loop initialization and the value stored opaquely? or is Funct(I) evaluated at the start of each iteration?

There is no right answer, these are options that a language designer must weigh up and choose.

Given the prevalence of passing structures in C on MCUs as almost always by address, it makes sense to recognize that and why it happens and then say "OK, well since whenever we pass a structure into some function in C on say STM32 devices, we always pass its address for the performance gain and because the callee will likely update the structure, lets recognize that and relieve the developer from having to always pass a pointer or pass the address".

We could even declare the datum itself with a "pass as" attribute, is that a good or bad idea? That way we can declare some structure to some concept like an A/D Converter as inherently always passed by val or ref, then there is no pass by convention, instead the means of passing is part of the type...just sayin...I've not thought about this at all, just throwing it out there, suggest, propose ideas, first, evaluate their utility later...I love ideas, I do not defer to orthodoxy, instead I like to hear all ideas, whether they be good or bad doesn't matter at first, we can always decide that later, this is lateral thinking, think freely, worry not, just generate ideas, encourage the breaking of convention, suggest the absurd even, but do not filter human thought prematurely, this is the essence of western European rationalism, the scientific revolution, there's nothing to worry about, get hung up about, this is just a discussion on the interet.

That's all I'm saying, sure, one can write code that abuses that or does it in a way that seems to undermine the principle, but we can do that with any language, one can take any language and write appalling code and then say "Look, I told you, that's a crap language".



Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 03, 2023, 07:25:42 pm
In a different discussion about another language the question of how best to exit or continue loops, particularly nested loops, came up for discussion.

What are people's thoughts about this kind of thing?

Title: Re: A microcontroller programming language
Post by: SiliconWizard on January 03, 2023, 07:39:49 pm
In a different discussion about another language the question of how best to exit or continue loops, particularly nested loops, came up for discussion.

What are people's thoughts about this kind of thing?

My take on this is named blocks, which generalizes loops.
Then to exit the block, 'exit xxx' or 'break xxx' or whatever syntax you like.

Example (again the syntax is just an example C-like, use whatever syntax you like - this is NOT C, so those tags would not be C goto labels, just block names):
Code: [Select]
Rows: for (i = 0; i < n; i++)
{
    Columns: for (j = 0; j < m; j++)
    {
        ...
        if (...)
            exit Rows;
    }
}

The block names or tags could be used for just any code block and not just loops. And again the syntax is just to show the idea, could be something else entirely. The idea is just to give code blocks a tag.
Simple and effective, and easy to read IMO. (Obviously you can do the same in C using goto and labels, but this isn't as readable IMO, and the above may allow better static analysis and better optimization.)
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 03, 2023, 07:48:19 pm
In a different discussion about another language the question of how best to exit or continue loops, particularly nested loops, came up for discussion.

What are people's thoughts about this kind of thing?

My take on this is named blocks, which generalizes loops.
Then to exit the block, 'exit xxx' or 'break xxx' or whatever syntax you like.

Example (again the syntax is just an example C-like, use whatever syntax you like - this is NOT C, so those tags would not be C goto labels, just block names):
Code: [Select]
Rows: for (i = 0; i < n; i++)
{
    Columns: for (j = 0; j < m; j++)
    {
        ...
        if (...)
            exit Rows;
    }
}

The block names or tags could be used for just any code block and not just loops. And again the syntax is just to show the idea, could be something else entirely. The idea is just to give code blocks a tag.
Simple and effective, and easy to read IMO. (Obviously you can do the same in C using goto and labels, but this isn't as readable IMO, and the above may allow better static analysis and better optimization.)

Yes that seems reasonable, the syntax <identifier><colon> is used by PL/I and C, used in PL/I to name procedures and functions in fact.

Its a bit disjointed though, in both C and PL/I.

In PL/I a procedure is named using a label:

Code: [Select]

sort_data:
   procedure (arg);
   // stuff
   end;

Yet PL/I does not use that form for naming declared items, C too only uses <identifier><colon> for label and nowhere else.

Of course in the example above "goto sort_data" would not compile because it knows that name is not a simple label.

PL/I also lets you break out of nested loops by referring to a label that prefixes some outer loop, all in all the use of label notation seems a bit odd to me in C and PL/I.

Is there another way? a "better" notation for labels?

Any easy thing to do would be a "label" keyword:

Code: [Select]

label outer_loop;

loop while (a > b)
   // stuff

   if (X) then
      leave outer_loop; // break in C
end;


But a keyword followed by the name? seems long winded...

I discovered that historically the <identifier><colon> form dates back to at least Algol, PL/I got it from that language and C got it from PL/I.

Ada uses <<label_name>> as a label.

Perhaps (since braces are not used by this new language, the 'draft' grammar) one could use { and } ?

Code: [Select]

{main_loop}

   loop
      // stuff
      if (a > b) then
         leave main_loop;
   end;

In that example {main_loop} visually stands out well, it is easy to see it's not some keyword or part of some ordinary statement, a : looks a lot like a ; too when perusing code, it also seems slightly better than Ada's << and >> IMHO...
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 05, 2023, 02:33:49 pm
Yes, all args are passed by ref. To achieve pass by value (that is, pass a ref to a temp copy) one just surrounds the arg in parens:

omfg. I am without words.

You just threw away a huge chunk of performance, as well as compatibility with all existing libraries and OS interfaces.

Ha! but not without expletives!

OK since you do want to discuss this with me, let me respond.

I'm referring to the general policy for arg passing, ultimately an implementation could do this in various ways (pass in a register for example). I'm not really referring to the implementation but to the semantics, I don't think I made that clear.

So "pass by reference" here is meant to indicate that the callee can change something in the datum that's been passed. That's the default convention, and if the the caller want's to prevent a callee from changing the value of an arg, they'd simply make that arg an expression with ( and ).

This leads to increased performance because unless the developer wants to pass by value, they can write the code and just let args pass by ref, meaning passing a bunch of structures, strings, arrays or combinations of these will entail nothing more than pushing a pointer onto the stack!

Ugh.

Code: [Select]
procedure foo(i : integer)
  var i : integer = 1;
  procedure bar(j : integer)
    i := i + 1;
    j := j + 1
  end
  bar(i);
  print(i)
end

What is printed?

Run it and it and see, then tell us the answer and then explain what the relevance is of asking this question? is it an attempt to make some "point"? well then state your case, state your proposition whatever it is.

I can not run it, sir.

This is my best guess of a possible syntax for YOUR proposed programming language, which you have not provided us with a compiler for.

My proposition is that your feature of everything is pass by reference not only sucks in general, but is especially nasty when combined with your feature of nested functions.

I'd be willing to create a runnable parser for the language that you can feed source code into and explore. There is no compiler yet, just a grammar and parser that proves that the grammar recognizes valid source code. If anyone would like to see that and explore it I'd  be willing to set aside some time to create that for general use.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 05, 2023, 06:24:18 pm
Evaluation order is a very important aspect of a programming language, C and C++ are very sloppy in this regard so I think this should be defined, not left to compiler writers to decide.

Java and C# are more formal, strict so I think a new language should follow their example which is left-to-right both within expressions and argument lists.

Short circuit evaluation too can be supported by dedicated operators, & and | for "evaluate" both sides and ?& and ?| for "evaluate the right only if the left has some value".

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 05, 2023, 07:00:50 pm
FYI - The language has a name now, it has been named "Imperium" after the Latin term for "control", source files are suffixed .ipl - Imperium Programming Language.

Title: Re: A microcontroller programming language
Post by: newbrain on January 05, 2023, 10:02:39 pm
FYI - The language has a name now, it has been named "Imperium" after the Latin term for "control", source files are suffixed .ipl - Imperium Programming Language.
Be careful, this might cost you a life sentence in Azkaban.
Title: Re: A microcontroller programming language
Post by: tggzzz on January 05, 2023, 10:20:52 pm
FYI - The language has a name now, it has been named "Imperium" after the Latin term for "control", source files are suffixed .ipl - Imperium Programming Language.

Usually the name is chosen to reflect a language's principal characteristics and behaviour.

Congratulations on making this important advance which will, no doubt about it, speed up progress.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 05, 2023, 11:08:23 pm
FYI - The language has a name now, it has been named "Imperium" after the Latin term for "control", source files are suffixed .ipl - Imperium Programming Language.

Usually the name is chosen to reflect a language's principal characteristics and behaviour.

Congratulations on making this important advance which will, no doubt about it, speed up progress.

There were practical reasons for selecting a name, many files are named after the language, the Antlr4 grammar has a name, the tools I use to build the grammar, the generated parser etc, it was a bit messy just having it name "lang". Documentation too needs to refer to the name, so although it might seem superficial to you it was not, like all not trivial software projects organization and consistency are important.

Since this language will give one greater, more refined control over a machine and its resources than crippled antiquated "C" it's an appropriate name, control is important, no more of this praying as we deliver code, praying that some buffer overrun doesn't crash the car or some array out of bounds doesn't reset the heart monitor or some unanticipated evaluation order does not shutdown the air conditioning.

Title: Re: A microcontroller programming language
Post by: cfbsoftware on January 06, 2023, 06:50:41 am
Here's a condensed list of important goals for a new language suitable for writing code for MCUs:

1. Extensible grammar (new keywords easily added over time, 100% backward compatibility guaranteed)
2. Elimination of pointer arithmetic.
3. Multidimensional arrays with variable upper/lower bounds.
4. An advanced preprocessor.
5. Support true 'offsets' in addition to conventional 'pointers'.
6. Supported nested procedures.
7. Support exceptions/conditions
8. Support 'builtin' and 'intrinsic' as fundamental language concepts.
9. Abstraction of interop - use a declarative means of interfacing to code written by other languages.
10. Support 'bit' as a data type.
11. Support flexible alignment, padding and ordering of structure/array members.
12. Support endianness operations for converting a contiguous memory block from big/little to little/big endian.
13. Support multiple 'heaps' and facilitate the freeing of all allocations within a heap by simply resetting the heap.
14. Innate support for coroutines.
15. Support decimal, octal, binary and hex numeric literals.
16. Support fixed point arithmetic in decimal/binary base.
17. Remain vigilante, aware of other old and new languages like Ada, Rust, Zig, Hare, Odin etc, where are these weak? where are they strong?

Is there anything fundamental missing from this list ? (I recognize that some want to talk about threading, synchronization and multiple cores, these are important but perhaps best set aside for now, to be revisited later).

It would be interesting to also examine various examples of C based solutions to certain MCU problems to see how a new language could improve the implementation of that code.

I'd like to also remind some here that all/any of these topics are up for discussion, I am not only concerned with grammar, if anyone wants to discuss any of these aspects then please be my guest, lets explore and move forward...
I have been promoting and supporting the use of Oberon-07 (https://people.inf.ethz.ch/wirth/Oberon/Oberon07.Report.pdf) as a language for programming ARM microcontrollers since 2008. The compiler and IDE is called Astrobe. I have checked it against your list of required features as follows:

1. Yes. See my previous post (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4581952/#msg4581952) in this discussion.
2. Yes.
3. Partial. Multidimensional arrays, but fixed lower bounds of zero.
4. No.
5. Absolute addressing is implemented using builtin functions ADR, GET, and PUT.
6. Yes.
7. Yes.
8. No.
9. No.
10. The datatype SET allows bit manipulation as described in Niklaus Wirth's 2007 paper:
SET: A neglected data type, and its compilation for the ARM (https://people.inf.ethz.ch/wirth/Oberon/SETs.pdf)
11. No. Unpacking / packing of structures where alignment is an issue is facilitated by builtin bitfield insert and extract inline functions.
12. No.
13. Single heap with customisable allocation / deallocation.
14. Not a language feature (unlike Oberon's predecessor Modula-2)
15. Decimal, hex and char numeric literals.
16. IEEE Floating point only.

Title: Re: A microcontroller programming language
Post by: DiTBho on January 06, 2023, 12:32:03 pm
I have been promoting and supporting the use of Oberon-07 as a language for programming ARM microcontrollers since 2008. The compiler and IDE is called Astrobe.

Frankly I think you have to write some real-life complex applications for the industry before you can say a single word with rational pearl of wisdom.

What have you seen written in Oberon?
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 06, 2023, 01:03:12 pm
Here's a condensed list of important goals for a new language suitable for writing code for MCUs:

1. Extensible grammar (new keywords easily added over time, 100% backward compatibility guaranteed)
2. Elimination of pointer arithmetic.
3. Multidimensional arrays with variable upper/lower bounds.
4. An advanced preprocessor.
5. Support true 'offsets' in addition to conventional 'pointers'.
6. Supported nested procedures.
7. Support exceptions/conditions
8. Support 'builtin' and 'intrinsic' as fundamental language concepts.
9. Abstraction of interop - use a declarative means of interfacing to code written by other languages.
10. Support 'bit' as a data type.
11. Support flexible alignment, padding and ordering of structure/array members.
12. Support endianness operations for converting a contiguous memory block from big/little to little/big endian.
13. Support multiple 'heaps' and facilitate the freeing of all allocations within a heap by simply resetting the heap.
14. Innate support for coroutines.
15. Support decimal, octal, binary and hex numeric literals.
16. Support fixed point arithmetic in decimal/binary base.
17. Remain vigilante, aware of other old and new languages like Ada, Rust, Zig, Hare, Odin etc, where are these weak? where are they strong?

Is there anything fundamental missing from this list ? (I recognize that some want to talk about threading, synchronization and multiple cores, these are important but perhaps best set aside for now, to be revisited later).

It would be interesting to also examine various examples of C based solutions to certain MCU problems to see how a new language could improve the implementation of that code.

I'd like to also remind some here that all/any of these topics are up for discussion, I am not only concerned with grammar, if anyone wants to discuss any of these aspects then please be my guest, lets explore and move forward...
I have been promoting and supporting the use of Oberon-07 (https://people.inf.ethz.ch/wirth/Oberon/Oberon07.Report.pdf) as a language for programming ARM microcontrollers since 2008. The compiler and IDE is called Astrobe. I have checked it against your list of required features as follows:

1. Yes. See my previous post (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4581952/#msg4581952) in this discussion.
2. Yes.
3. Partial. Multidimensional arrays, but fixed lower bounds of zero.
4. No.
5. Absolute addressing is implemented using builtin functions ADR, GET, and PUT.
6. Yes.
7. Yes.
8. No.
9. No.
10. The datatype SET allows bit manipulation as described in Niklaus Wirth's 2007 paper:
SET: A neglected data type, and its compilation for the ARM (https://people.inf.ethz.ch/wirth/Oberon/SETs.pdf)
11. No. Unpacking / packing of structures where alignment is an issue is facilitated by builtin bitfield insert and extract inline functions.
12. No.
13. Single heap with customisable allocation / deallocation.
14. Not a language feature (unlike Oberon's predecessor Modula-2)
15. Decimal, hex and char numeric literals.
16. IEEE Floating point only.

Thanks for posting this, I'm definitely going to explore this, SETS looks very interesting, I'll talk about this soon...
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 06, 2023, 01:17:32 pm
I have been promoting and supporting the use of Oberon-07 as a language for programming ARM microcontrollers since 2008. The compiler and IDE is called Astrobe.

Frankly I think you have to write some real-life complex applications for the industry before you can say a single word with rational pearl of wisdom.

What have you seen written in Oberon?

This illogical reasoning needs to be dispelled, the validity of propositions about the world DOES NOT DEPEND on the experiences, skills, personality of the person stating said proposition.

He can only be shown to be wrong by you proving him wrong, so present concrete source code, evidence, PROOF else your claims are without substance.

You must prove him wrong, with facts, logic, proof otherwise we end up bickering over anecdotes, considering were in an engineering thread, discussing formal rule based systems, the regularity with which some here post emotionally or vindictively is incredible, lets have more rigor and act like the professional thinkers we strive to be.

Title: Re: A microcontroller programming language
Post by: DiTBho on January 06, 2023, 01:54:32 pm
I am willing to re-write all MPC860 firmware in Oberon. Then I will try to re-write some driver for ucOS/2 in Oberon. To check how a language is good/bad for your business needs.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 06, 2023, 02:16:16 pm
Programming languages are not simply something we express our ideas and thoughts in, they directly influence how we think and structure solutions to problems, there is a mutual "feedback" loop present between our minds and the language.

If one uses C daily for years and years then that repeated mental activity, over and over and over begins to define how we think about our problems, this is well demonstrated by taking any experienced imperative language programmer and teaching them a functional language, it's often hard, a struggle not because the person is a dummy but because they've unwittingly conditioned their minds to see things through some narrow window, armed with a functional language they feel constrained, tied down, like they were trying to mend a watch while wearing boxing gloves.

This is because they need to think differently when using the functional language, it is therefore not the language they struggle with but themselves, their own minds.

Every seasoned experienced C programmer here who's never really used a functional language to write non-trivial code, should spend some time looking at say Haskell or F# not because its "better than C" but because it is the best way to see how conditioned their minds have become.

People don't like change by nature, but fear of change is a huge roadblock to intellectual progress, one cannot improve things without changing things.





Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 06, 2023, 02:29:40 pm
I have been promoting and supporting the use of Oberon-07 as a language for programming ARM microcontrollers since 2008. The compiler and IDE is called Astrobe.

Frankly I think you have to write some real-life complex applications for the industry before you can say a single word with rational pearl of wisdom.

What have you seen written in Oberon?

See page 30: https://www.xilinx.com/publications/archives/xcell/Xcell91.pdf (https://www.xilinx.com/publications/archives/xcell/Xcell91.pdf)

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 06, 2023, 02:45:31 pm
Here's a condensed list of important goals for a new language suitable for writing code for MCUs:

1. Extensible grammar (new keywords easily added over time, 100% backward compatibility guaranteed)
2. Elimination of pointer arithmetic.
3. Multidimensional arrays with variable upper/lower bounds.
4. An advanced preprocessor.
5. Support true 'offsets' in addition to conventional 'pointers'.
6. Supported nested procedures.
7. Support exceptions/conditions
8. Support 'builtin' and 'intrinsic' as fundamental language concepts.
9. Abstraction of interop - use a declarative means of interfacing to code written by other languages.
10. Support 'bit' as a data type.
11. Support flexible alignment, padding and ordering of structure/array members.
12. Support endianness operations for converting a contiguous memory block from big/little to little/big endian.
13. Support multiple 'heaps' and facilitate the freeing of all allocations within a heap by simply resetting the heap.
14. Innate support for coroutines.
15. Support decimal, octal, binary and hex numeric literals.
16. Support fixed point arithmetic in decimal/binary base.
17. Remain vigilante, aware of other old and new languages like Ada, Rust, Zig, Hare, Odin etc, where are these weak? where are they strong?

Is there anything fundamental missing from this list ? (I recognize that some want to talk about threading, synchronization and multiple cores, these are important but perhaps best set aside for now, to be revisited later).

It would be interesting to also examine various examples of C based solutions to certain MCU problems to see how a new language could improve the implementation of that code.

I'd like to also remind some here that all/any of these topics are up for discussion, I am not only concerned with grammar, if anyone wants to discuss any of these aspects then please be my guest, lets explore and move forward...
I have been promoting and supporting the use of Oberon-07 (https://people.inf.ethz.ch/wirth/Oberon/Oberon07.Report.pdf) as a language for programming ARM microcontrollers since 2008. The compiler and IDE is called Astrobe. I have checked it against your list of required features as follows:

1. Yes. See my previous post (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4581952/#msg4581952) in this discussion.
2. Yes.
3. Partial. Multidimensional arrays, but fixed lower bounds of zero.
4. No.
5. Absolute addressing is implemented using builtin functions ADR, GET, and PUT.
6. Yes.
7. Yes.
8. No.
9. No.
10. The datatype SET allows bit manipulation as described in Niklaus Wirth's 2007 paper:
SET: A neglected data type, and its compilation for the ARM (https://people.inf.ethz.ch/wirth/Oberon/SETs.pdf)
11. No. Unpacking / packing of structures where alignment is an issue is facilitated by builtin bitfield insert and extract inline functions.
12. No.
13. Single heap with customisable allocation / deallocation.
14. Not a language feature (unlike Oberon's predecessor Modula-2)
15. Decimal, hex and char numeric literals.
16. IEEE Floating point only.

I was reading this doument: https://www.astrobe.com/OberonGuide.pdf (https://www.astrobe.com/OberonGuide.pdf)

In there it defines SET to be "The sets of integers between 0 and 31".

What does this actually mean? does it mean that: 0100 0000 0000 0000 0010 1011 1101 0101 implements a set that contains the integers {0,2,4,6,7,8,9,11,13,30}?

(assuming the LSB is the rightmost bit).

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 07, 2023, 12:08:39 am
Should anyone be curious the Github repository for this work is here - imperium repository (https://github.com/Steadsoft/imperium).

The formal grammar and tooling now facilitate support for multiple cultures, it is trivial to now create keyword sets (https://github.com/Steadsoft/imperium/blob/main/grammar/Antlr/imperium.keywords.json) for other cultures providing tokens can be defined for them in that language. There's obviously English and also recently Dutch (kindly contributed by the lead engineer of the Citrine programming language), (http://citrine-lang.org/) French is something I expect to add very soon too.

I'm also very interested in some of the stuff from earlier "Wirthian" languages, specifically the concept of "SETS" as seen in Oberon and earlier languages that have recently been brought to my attention.

Oberon itself is proving to be very interesting, here's a 2015 in-depth technical article from Xilinx about an FPGA implementation (https://www.xilinx.com/publications/archives/xcell/Xcell91.pdf) of Oberon by Nicklaus Wirth for those interested (jump to page 30).

Title: Re: A microcontroller programming language
Post by: cfbsoftware on January 07, 2023, 03:34:04 am
Frankly I think you have to write some real-life complex applications for the industry before you can say a single word with rational pearl of wisdom.

What have you seen written in Oberon?
I am continually surprised by what some of our Astrobe users are managing to do with Oberon. Here are a couple of comments:

Quote
Making good progress with the EFM32 parts. I've started using the internal EFM32 DMA controllers to manage up to 4 streams of parallel data at 2 mbit / sec plus all the other card io that goes on.

The GD32F130s are working out well for the smaller simple modules where we use many at a time ( Could be anywhere from 50 to 500 or more in a console ). The GD32F130s all have 485 data ports. A custom bootloader can update run-time firmware using the 485 data ports ( @ 2 mbit / sec ). Program update data sent from the EFM32 controllers. Can update similar type modules in parallel that way, so don't have to update them one at a time. The bootloader is installed during initial card test using a Segger & Tag-Connect footprint on the card. The bootloader is written in Oberon as well. Bootloader is under 16K, so leaves 48K for application which is lots for those modules.

Quote
Our systems are now installed at six nursing homes. This Thursday, we will install a system at a home for severe mental disability patient. Four more nursing homes want a pilot installation. It seems we have struck a nerve, especially with the younger nurses, who really embrace the support that this system is providing them.

Title: Re: A microcontroller programming language
Post by: cfbsoftware on January 07, 2023, 03:47:14 am
I was reading this doument: https://www.astrobe.com/OberonGuide.pdf (https://www.astrobe.com/OberonGuide.pdf)

In there it defines SET to be "The sets of integers between 0 and 31".

What does this actually mean? does it mean that: 0100 0000 0000 0000 0010 1011 1101 0101 implements a set that contains the integers {0,2,4,6,7,8,9,11,13,30}?

(assuming the LSB is the rightmost bit).
Correct. Some time ago Elektor Magazine published an article I wrote titled: Easy Sceptre Programming with Oberon-07. The Sceptre was an ARM LPC2148 microcontroller development board designed by Elektor staff. I described one use of SETs there:

The SET data type: testing, setting and clearing bits
Much of the low-level programming of the Sceptre’s LPC2148 involves testing, setting and clearing specific bits of the memory-mapped registers. This is where Oberon’s built-in SET data type is very useful.

For example, if you have LEDs connected to pins P0.13, P0.14 and P0.15 you can light them the traditional way using hexadecimal notation:

SYSTEM.PUT (LPC.IOSET, 0E000H)

Alternatively, if you prefer, using a SET constant you can say:

SYSTEM.PUT (LPC.IOSET, { 13..15 })

The ARM7 instructions generated are identical for both cases.

To help you when working from examples using hexadecimal constants, the Astrobe menu item “Tools > Convert Hex to SET” converts hem to the equivalent Oberon-07 SET constant ready to be pasted into your code.
Title: Re: A microcontroller programming language
Post by: SiliconWizard on January 07, 2023, 04:13:32 am
The SET type of Oberon is certainly a nice way of defining binary constants, much more readable than writing in binary such as 0b001010111001
Title: Re: A microcontroller programming language
Post by: SiliconWizard on January 07, 2023, 04:15:19 am
Frankly I think you have to write some real-life complex applications for the industry before you can say a single word with rational pearl of wisdom.

What have you seen written in Oberon?
I am continually surprised by what some of our Astrobe users are managing to do with Oberon.

As Wirth said, complex systems should be designed with simple tools. He also added that he was aware of his opinion not being very popular.
Title: Re: A microcontroller programming language
Post by: MIS42N on January 07, 2023, 04:22:34 am
The SET data type: testing, setting and clearing bits
Much of the low-level programming of the Sceptre’s LPC2148 involves testing, setting and clearing specific bits of the memory-mapped registers. This is where Oberon’s built-in SET data type is very useful.

For example, if you have LEDs connected to pins P0.13, P0.14 and P0.15 you can light them the traditional way using hexadecimal notation:

SYSTEM.PUT (LPC.IOSET, 0E000H)

Alternatively, if you prefer, using a SET constant you can say:

SYSTEM.PUT (LPC.IOSET, { 13..15 })


That is rather neat. It beats ( 1<<13 | 1<<14 | 1<<15 ) for conciseness, and even more readable as long as one recognises the curly braces as bit numbers. I assume the use of SET to define bits and the use of SET in IOSET is just coincidental. I could imagine a different architecture with LPC.IOCLR, { 13..15 } or similar
Title: Re: A microcontroller programming language
Post by: SiliconWizard on January 07, 2023, 04:29:16 am
It's a set, so you can mix bit intervals such as: { 0, 3, 5..7, 12..13, 15 }, that's sweet indeed.
Title: Re: A microcontroller programming language
Post by: cfbsoftware on January 07, 2023, 09:45:24 am
That is rather neat. It beats ( 1<<13 | 1<<14 | 1<<15 ) for conciseness, and even more readable as long as one recognises the curly braces as bit numbers. I assume the use of SET to define bits and the use of SET in IOSET is just coincidental. I could imagine a different architecture with LPC.IOCLR, { 13..15 } or similar
Exactly. IOSET is the name used in the LPC2xxx microcontroller manual. The Blinker example module has a constant declaration:
Code: [Select]
  CONST
    (* led(s) connected to pin P0.13, P0.14 and P0.15 *)
    ledBits = {13..15};
... and an infinite main loop:
Code: [Select]
  WHILE TRUE DO
    SYSTEM.PUT(LPC.IOCLR, ledBits);
    Timer.MSecDelay(500);
    SYSTEM.PUT(LPC.IOSET, ledBits);
    Timer.MSecDelay(500)
  END
IOCLR and IOSET are absolute memory addresses declared as constants in a separate, imported, LPC module:
Code: [Select]
  IOSET* = 0E0028004H;
  IODIR* = 0E0028008H;
  IOCLR* = 0E002800CH;
The asterisk indicates that the values are exported from the module.
Title: Re: A microcontroller programming language
Post by: cfbsoftware on January 07, 2023, 09:54:10 am
Frankly I think you have to write some real-life complex applications for the industry before you can say a single word with rational pearl of wisdom.

What have you seen written in Oberon?
I am continually surprised by what some of our Astrobe users are managing to do with Oberon.

As Wirth said, complex systems should be designed with simple tools. He also added that he was aware of his opinion not being very popular.

Yes indeed. I should also have mentioned that Wirth's Oberon compilers are entirely written in Oberon. They make good regression test cases ;)
Title: Re: A microcontroller programming language
Post by: DiTBho on January 07, 2023, 11:37:50 am
Some time ago Elektor Magazine published an article I wrote titled: Easy Sceptre Programming with Oberon-07. The Sceptre was an ARM LPC2148 microcontroller development board designed by Elektor staff.

is it possible, somehow, to get a copy?
Title: Re: A microcontroller programming language
Post by: Siwastaja on January 07, 2023, 12:36:01 pm
That is rather neat. It beats ( 1<<13 | 1<<14 | 1<<15 ) for conciseness

But you can write (0b111<<13), this is something I do all the time.
Title: Re: A microcontroller programming language
Post by: DiTBho on January 07, 2023, 12:52:29 pm
half technology, half religion  :-//

Title: Re: A microcontroller programming language
Post by: DC1MC on January 07, 2023, 01:13:01 pm
Some time ago Elektor Magazine published an article I wrote titled: Easy Sceptre Programming with Oberon-07. The Sceptre was an ARM LPC2148 microcontroller development board designed by Elektor staff.

is it possible, somehow, to get a copy?

Sure:
https://www.elektormagazine.de/labs/easy-sceptre-programming-with-oberon-07 (https://www.elektormagazine.de/labs/easy-sceptre-programming-with-oberon-07)

https://www.astrobe.com/Oberon.htm (https://www.astrobe.com/Oberon.htm)
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 07, 2023, 01:46:00 pm
That is rather neat. It beats ( 1<<13 | 1<<14 | 1<<15 ) for conciseness, and even more readable as long as one recognises the curly braces as bit numbers. I assume the use of SET to define bits and the use of SET in IOSET is just coincidental. I could imagine a different architecture with LPC.IOCLR, { 13..15 } or similar
Exactly. IOSET is the name used in the LPC2xxx microcontroller manual. The Blinker example module has a constant declaration:
Code: [Select]
  CONST
    (* led(s) connected to pin P0.13, P0.14 and P0.15 *)
    ledBits = {13..15};
... and an infinite main loop:
Code: [Select]
  WHILE TRUE DO
    SYSTEM.PUT(LPC.IOCLR, ledBits);
    Timer.MSecDelay(500);
    SYSTEM.PUT(LPC.IOSET, ledBits);
    Timer.MSecDelay(500)
  END
IOCLR and IOSET are absolute memory addresses declared as constants in a separate, imported, LPC module:
Code: [Select]
  IOSET* = 0E0028004H;
  IODIR* = 0E0028008H;
  IOCLR* = 0E002800CH;
The asterisk indicates that the values are exported from the module.

This is a great notation, I'm very interested in adopting this into the IPL grammar. This is exactly the kind of hardware "friendly" stuff I've been seeking, stuff that helps in real life problems, pragmatic. In IPL (taken from PL/I) there are "bit strings", arbitrary length sequences of bits (either aligned or not) that could be manipulated by this "set" notation, it's very attractive.

There's no reason too, why one could not have this arbitrarily defined, no need to be concerned that it's 32 bits or 64 or anything:

Code: [Select]

dcl interrupt_control bit(45); // usual PL/I syntax

interrupt_control = {}; // set all bits OFF

interrupt_control = {0..4,19..30}; // Wirthian notation


IPL does not yet use { or } or .. either and so this is pretty straightforward to add to the grammar, it seems to me that this is primarily a notation, its implementation, semantics are relatively simple.

I have questions though, look:

Code: [Select]

flags = {0..5, 7..9};


That effectively sets bits 0, 1, 2, 3, 4, 5 and 7, 8, 9 to "on". How would I express "turn off bit 3 in flags"?

Code: [Select]

flags = flags - {3};


would that be it? (in Oberon anyway).

PL/I and Imperium too, support the ability to access a "bit string" via a subscript:

Code: [Select]

dcl control_reg bit(32);

control_reg = get_reg(X);

control_reg(4) = ON;
control_reg(19) = ON;
control_reg(28) = OFF;


But setting several ON/OFF at once isn't really supported, there's no short notation for it anyway.

With Wirth's notation we could code:

Code: [Select]

dcl control_reg bit(32);

control_reg = get_reg(X);

control_reg = control_reg + {4,19} - {28};


So I'm seeing this as an additional notation for setting and unsetting bits, not a change or alternative way to implement bit strings (as they are termed) but a very neat additional notation one can use.


Title: Re: A microcontroller programming language
Post by: SiliconWizard on January 07, 2023, 08:46:24 pm
That is rather neat. It beats ( 1<<13 | 1<<14 | 1<<15 ) for conciseness

But you can write (0b111<<13), this is something I do all the time.

Yes, that's clearer.
And if you define your bits as macros, admittedly writing '(THISBIT | THISOTHERBIT)' or '{THISBIT , THISOTHERBIT}' is not a lot different.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 07, 2023, 10:25:02 pm
After much painstaking testing I have proven the multilingual support for this, it was very confusing to test and debug due to the way different editors store accented characters and Unicode, it was almost maddening but is absolutely fine now.

These two code fragments generate the same logical parse tree, one parsed in English mode, the other in French mode:

Code: [Select]

procédé French (X)

déclarer counter binaire(15);

si counter > 0 ensuite
   appeler escaper;
autre
   retour;
fin;

fin;

and

Code: [Select]
procedure English (X)

declare counter binary(15);

if counter > 0 then
   call escaper;
else
   return;
end;

end;

This was actually small engineering effort (but a struggle due to the confusion that can come up when working with UTF-8 and various different editors etc) and is something that can be routinely included going forward, sets of keywords can be added for any language (within reason) and the code using those keywords guaranteed to parse, this is one of the benefits of having no reserved words.

Source code can be transformed from one language into another easily by a utility that alters only keywords, then any user defined name that might just happen to be a keyword in that target language, are fine, they do not and cannot break the parse!

So auto converting this:

Code: [Select]


procedure English (X)

declare counter binary(15);

if counter > 0 then
   call retour; // Call the retour module to review the map's touring data.
else
   return;
end;

end;

into French:

Code: [Select]
procédé French (X)

déclarer counter binaire(15);

si counter > 0 ensuite
   appeler retour; // Call the retour module to review the map's touring data.
autre
   retour;
fin;

fin;

Works, it parses fine (in French mode), even though the English developer that created the original code unknowingly used an identifier "retour" (for the name of some procedure) that just so happens to be a French keyword (for "return"). This is an extremely powerful capability potentially, and no conventional programming language you're likely to encounter will have this ability.





Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 08, 2023, 04:47:00 pm
Here's a richer example showing how keywords may contain multiple terms, space separated terms, each of these when parsed generates an identical logical parse tree:

Code: [Select]
procedure English (X)

declare counter binary(15);

if counter > 0 then
   call retour;
else
   return;
end;

loop while (a > b)
   go to place;
end;

end;

Code: [Select]
procédé French (X)

déclarer counter binaire(15);

si counter > 0 ensuite
   appeler retour;
autre
   retour;
fin;

boucle tandis que (a > b)
   aller à place;
fin;

fin;

The grammar supports alternate keyword forms too, so in the English code we can write "go to" or "goto", so that flexibility is available if desired for some languages.



Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 08, 2023, 05:34:58 pm
German:

Code: [Select]
routine German (X)

deklarieren counter binär(15);

wenn counter > 0 dann
   aufrufen retour;
anders
   rückkehr;
ende;

wiederholen solang (a > b)
   gehen place;
ende;

ende;
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 09, 2023, 07:26:30 pm
Russian:

Code: [Select]

процедура Russian (X)

   объявлять counter бинарный(15);

   counter = FACE B034 D85F BAD7:H;

   если counter > 0 затем
      вызов retour;
   еще
      возвращаться;
   конец;

   повторять в то время как (a > b)
ходят в place;
   конец;

конец;

Title: Re: A microcontroller programming language
Post by: ataradov on January 09, 2023, 07:45:18 pm
Russian one reads very poorly. My head would hurt if I had to read that all day long. Russian is very bad for programming languages, since words change a lot with gender, tense, plurality.

Also, "вызов" is a noun, to be consistent it should be "вызвать" (verb). Or "возвращаться" should be "возврат". But not both at the same time. And for a proper sentence structure "затем" should be "то".

And the whole idea of programming languages with different "reserved" words is pretty bad. It completely shuts down international teamwork. People don't do this not because it is impossible, but because it is impractical and nobody wants it.

There is a huge Russian accounting system from a company called 1C. They used Russian language in their scripting system. There was so much demand that they added English support. And the software is very specific for Russian market and tax laws, so there are no users outside of Russia and may be former Soviet republics.

Also, you would have to constantly switch the layout to type it, since parts of it (like constants) are still English only. This is more keystrokes, just to get a less readable result.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 09, 2023, 08:06:32 pm
Russian one reads very poorly. My head would hurt if I had to read that all day long. Russian is very bad for programming languages, since words change a lot with gender, tense, plurality.

Also, "вызов" is a noun, to be consistent it should be "вызвать" (verb). Or "возвращаться" should be "возврат". But not both at the same time. And for a proper sentence structure "затем" should be "то".

Thank you, I'll factor your guidance into the dictionary, much appreciated.

And the whole idea of programming languages with different "reserved" words is pretty bad. It completely shuts down international teamwork. People don't do this not because it is impossible, but because it is impractical and nobody wants it.

Well I'm sure we could argue some of the claims all day, there are numerous languages that cater for non-English speakers take Citrine for example as one of the more prevalent, also Algol-68 supported Russian (https://ieeexplore.ieee.org/abstract/document/7032965). I don't see how any of these claims about teamwork and being impractical can be justified, the grammar doesn't care at all about the keyword vocabulary, it is unaware, completely unaware. The design is sound, one can take source code written with any vocabulary and instantly convert it to any other 100% automatically, I can envisage an IDE having a dropdown to select rendered language and a "Save As" to save code in any desired supported language, no problem whatsoever.

Being able to take some complex logic written in English and automatically, instantly render it in Russian, Hebrew, Spanish, Serbian or whatever language one wants, can't honestly be described "bad" or "impractical".

If a team wanted to work in English then they'd just establish that as a team standard, changing the keyword vocabulary is no different to changing the IDE's font or colors, its an irrelevant detail.

There is a huge Russian accounting system from a company called 1C. They used Russian language in their scripting system. There was so much demand that they added English support. And the software is very specific for Russian market and tax laws, so there are no users outside of Russia and may be former Soviet republics.

Also, you would have to constantly switch the layout to type it, since parts of it (like constants) are still English only. This is more keystrokes, just to get a less readable result.

Picking a bad example of multi-language support does not prove the general argument you're trying to make.

Quote
In English, Algol68's case statement reads case ~ in ~ out ~ esac. In Russian, this reads выб ~ в ~ либо ~ быв.

See (https://en.wikipedia.org/wiki/Non-English-based_programming_languages).

An English-only speaker and a Rusian-only speaker could work on the same code base and only ever see the code in their native language, I'd see only English keywords yet the Russian speaker would only see Russian, we'd never even care or know what language a co-worker preferred or used!

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 09, 2023, 08:25:15 pm
[attachimg=1]There, I made your changes, it took me two minutes Alex (updated the dictionary JSON (https://github.com/Steadsoft/imperium/blob/main/grammar/Antlr/imperium.keywords.json), regen the grammar keyword definitions, change the source code to match - do a test parse).

Code: [Select]
процедура Russian (X)

   объявлять counter бинарный(15);

   counter = FACE B034 D85F BAD7:H;

   если counter > 0 то
      вызвать retour;
   еще
      возврат;
   конец;

   повторять в то время как (a > b)
ходят в place;
   конец;

конец;


Parses fine, all good.
Title: Re: A microcontroller programming language
Post by: ataradov on January 09, 2023, 08:39:59 pm
Quote
Also, "вызов" is a noun, to be consistent it should be "вызвать" (verb). Or "возвращаться" should be "возврат". But not both at the same time.
You just flipped them. Now the first one is a verb and the second one is a noun.

Also, I just realized what "ходят в" is supposed to be goto.  It is far better to use "перейти к" or "переход к". "перейти" is a verb, "переход" is a noun. So, again, use whichever ones you want to use in other places.
Title: Re: A microcontroller programming language
Post by: tggzzz on January 09, 2023, 08:50:52 pm
And the whole idea of programming languages with different "reserved" words is pretty bad. It completely shuts down international teamwork. People don't do this not because it is impossible, but because it is impractical and nobody wants it.

I tend to agree.

How many languages are there? How many won't be supported?

In many of my jobs it has been normal to have several different nationalities in the team.
If multiple languages are allowed, will if( aRectangle.সূচনাকরুন() ) {...} be allowed? If not, why not?

Which language would be stored in the source code control system?
How would "diffs" be done?
Who will write the tool that allows reconciles code in Basque, Finnish, Malayalam, Arabic, Hebrew?
While we don't have any boustrophedonic languages any more, will the top-to-bottom languages be excluded?
And, of course there is the issue of pair programming? Which language wins out there?

How will IDEs cope when parsing code so they can provide ctrl-space style autocompletion?
Can a programmer add a method in Chinese to a class in Spanish with a superclass in Welsh?

Yup it is a whole can of worms. Makes APL's somewhat arbitrary symbols look like a sensible option.

Maybe there's a reason  the Algol experience hasn't been repeated!
Title: Re: A microcontroller programming language
Post by: DC1MC on January 09, 2023, 08:52:29 pm
So this wonder of modern programming language will it have a translator (for example if a German team will pick up after a Russian team) to switch back and forth of the human languages ? Or if one has to do a small change will have to do it in Russian or Chinese.

I have to say that, initially, I didn't have great expectations, many attempt were made and abandoned, but now I'm amazed, I am so amazed about this collection of useless, anti-pattern and destructive features that I hope this will become an actual language, to be used as an "a la carte" example of instant firing/ failing exam reason. I'm really baffled, is this some subtle trolling attempt or someone actually wants to create such a thing and considers it an improvement, I'm at a loss of words, ALL, but ABSOLUTE ALL features that could make a programing language unusable, impossible to lint, difficult to learn and use because of "flexible" keywords, abnormal syntax that doesn't follow any existing conventions, except for failing attempts decades ago, what else should it have to make it even more cumbersome and loathsome to use, mind boggles  |O ?
 
And this is supposed to be used for microcontrollers, good Lord, mon Dieu, boje moi, милостивый бог !!!

At this time I can only assume is a trolling attempt, magisterially executed, chapeau bas !!!

Cheers and (sincerely) good luck,
DC1MC

Title: Re: A microcontroller programming language
Post by: tggzzz on January 09, 2023, 09:01:25 pm
I'm really baffled, is this some subtle trolling attempt or someone actually wants to create such a thing

General principle: never attribute to malice that which can be explained by stupidity. Or incompetence. Or ego.

But there's no harm in idle speculation, particularly if contrary views are listened to  and understood rather than being dismir.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 09, 2023, 09:02:09 pm
Quote
Also, "вызов" is a noun, to be consistent it should be "вызвать" (verb). Or "возвращаться" should be "возврат". But not both at the same time.
You just flipped them. Now the first one is a verb and the second one is a noun.

Also, I just realized what "ходят в" is supposed to be goto.  It is far better to use "перейти к" or "переход к". "перейти" is a verb, "переход" is a noun. So, again, use whichever ones you want to use in other places.

Very helpful, all tweaked!

Thank you Alex.

Title: Re: A microcontroller programming language
Post by: DiTBho on January 09, 2023, 09:05:37 pm
Pure garbage!
Title: Re: A microcontroller programming language
Post by: DiTBho on January 09, 2023, 09:10:33 pm
Computer science is a branch of mathematics, so it's as idiotic as thinking about translating mathematical symbols into *something*.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 09, 2023, 09:11:21 pm
So this wonder of modern programming language will it have a translator (for example if a German team will pick up after a Russian team) to switch back and forth of the human languages ? Or if one has to do a small change will have to do it in Russian or Chinese.

The answers to these questions are rather clear if you take the time to read some of what I wrote.

I have to say that, initially, I didn't have great expectations, many attempt were made and abandoned, but now I'm amazed, I am so amazed about this collection of useless, anti-pattern and destructive features that I hope this will become an actual language, to be used as an "a la carte" example of instant firing/ failing exam reason.

You're too kind.

I'm really baffled, is this some subtle trolling attempt or someone actually wants to create such a thing and considers it an improvement, I'm at a loss of words...

If only that were true.

ALL, but ABSOLUTE ALL features that could make a programing language unusable, impossible to lint, difficult to learn and use because of "flexible" keywords, abnormal syntax that doesn't follow any existing conventions, except for failing attempts decades ago, what else should it have to make it even more cumbersome and loathsome to use, mind boggles  |O ?
 
And this is supposed to be used for microcontrollers, good Lord, mon Dieu, boje moi, милостивый бог !!!

At this time I can only assume is a trolling attempt, magisterially executed, chapeau bas !!!

Cheers and (sincerely) good luck,
DC1MC

What a surprisingly nice, if insincere ending to an otherwise excellent example of bitterness, bigotry and diatribe.



Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 09, 2023, 09:13:42 pm
Computer science is a branch of mathematics, so it's as idiotic as thinking about translating mathematical symbols into *something*.

No it isn't and nor is it a science.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 09, 2023, 09:36:01 pm
So this wonder of modern programming language will it have a translator (for example if a German team will pick up after a Russian team) to switch back and forth of the human languages ? Or if one has to do a small change will have to do it in Russian or Chinese.

You'll need to explain why an ability to optionally render source code into one of a variety of cultures, is such a frightening prospect to you.

This is a detail, in the case of Imperium it was low hanging fruit due to the absence of reserved words, it was deemed potentially valuable and cost almost nothing, any grammar that lacks reserved words has this capability for free so I decided to develop it and I'm pleased with the results.

It's a detail though, just as the fact that a compiler can typically target a variety of CPUs, is a detail, abstracted away.

The vocabulary rendered on screen is as immaterial as the colors or fonts one uses in their IDE, I don't care if you use bold purple for C keywords or italic blue, why would I care that you have that freedom? if you find one makes your more productive, improves your concentration then use it!

Likewise why would you care if some individual with little familiarity with English (not that "goto" and "declare" and "static" even have the same semantics found in an English dictionary for these words!) be able to see source code using terms and words that are innately meaningful and pronounceable in their native language?

If a Bulgarian speaker, new to computer programming can learn more quickly, develop better understanding, can experiment more meaningfully using Bulgarian keywords, why is that a bad thing to you?

Consider the word "declare" here's what an English dictionary says:

Quote
"to announce something clearly, firmly, publicly, or officially"

or "static"

Quote
"showing little change"

Yet in code it means "tell the compiler we want to create a variable and name it <whatever>" or "this variable is not situated in stack memory", this and many many other examples confront the non-English user. Not only must they "learn" English words, those English words aren't even used as they are used in day to day English! The above calls out "publicly" too !

Consider:

Quote

declare counter binary static private;


See? the English dictionary definition of "declare" calls out "public" yet they're expected to use the word "declare" for something "private" - how is that not confusing? or why label a variable as "static" when that means "not to change" ? Then consider we might have fifty, sixty or more terms, keywords, attributes, options and directives and so on - and you have the audacity to insist that situation is good for everyone? not exclusionary? everybody should work in English?

If you truly want to talk about programming languages then all these issues are part of the subject, so get used to it.
Title: Re: A microcontroller programming language
Post by: MIS42N on January 10, 2023, 01:29:29 am
Programming languages are called languages because they are. They are related to, but not one to one mapped to any human language. So translating the word 'declare' into a different human language is meaningless unless the users of that human language have an equivalent computer language where there is a one to one correspondence to the word 'declare'. Using the English dictionary definition of a word used in a programming language does not make sense. It is unfortunate that many words used in programming languages do have English meaning somewhat equivalent to their use in programming. Perhaps we'd be better divorcing the two, maybe the word 'gift' could be used instead of 'static'.

But maybe the flexibility proposed is useful. Each user could select a preprocessor that works two ways - converts source code to tokens, and tokens to source code. The source code is ephemeral, only exists on the user's screen. The transportable entity is the token code with one or more translation tables. A user can elect to use an existing translation table or add one.

Much of programming consists of operators and operands. In good programming the operands are given meaningful names in the language of the creator. It would be useful to translate the operands between users who have different human languages. I have encountered programs written by Germans where the intent is obscure until the operands are decoded. It would be convenient if I wished to work on the program to use the English equivalent of the operands but able to return the program to the creator with his own operands. Adding an English translation table to the original tokenised program would allow that.

Just a thought.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 03:18:44 am
And the whole idea of programming languages with different "reserved" words is pretty bad. It completely shuts down international teamwork. People don't do this not because it is impossible, but because it is impractical and nobody wants it.

I tend to agree.

How many languages are there? How many won't be supported?

In many of my jobs it has been normal to have several different nationalities in the team.
If multiple languages are allowed, will if( aRectangle.সূচনাকরুন() ) {...} be allowed? If not, why not?

Which language would be stored in the source code control system?
How would "diffs" be done?
Who will write the tool that allows reconciles code in Basque, Finnish, Malayalam, Arabic, Hebrew?
While we don't have any boustrophedonic languages any more, will the top-to-bottom languages be excluded?
And, of course there is the issue of pair programming? Which language wins out there?

How will IDEs cope when parsing code so they can provide ctrl-space style autocompletion?
Can a programmer add a method in Chinese to a class in Spanish with a superclass in Welsh?

Yup it is a whole can of worms. Makes APL's somewhat arbitrary symbols look like a sensible option.

Maybe there's a reason  the Algol experience hasn't been repeated!

APL's symbols weren't arbitrary read Iverson's Turing Award lecture, hopefully you'll learn something about this subject. Your confused questions make it clear you understand little about this subject.

Everything I've said about this feature is OPTIONAL just like any number of coding standards or conventions. There is no computational reason for language keywords to be restricted to English, the abstract concepts are not English, or Japanese or Hebrew, they are universal  concepts just like mathematical ideas, or APL ideas.

You can use Google translate right now to translate Russian or Norwegian articles into English, is that a bad thing? does that cause nothing but problems and confusion? do people try to mix multiple languages within the same article? think about what I'm saying to you.

Being able to translate source code from English to Spanish to Dutch to Russian with absolutely no impact on the code's meaning or runtime behavior is actually a hugely powerful capability, one you clearly cannot perceive or grasp, it goes right over your head.
Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 09:05:46 am
The vocabulary rendered on screen is as immaterial as the colors or fonts one uses in their IDE, I don't care if you use bold purple for C keywords or italic blue, why would I care that you have that freedom? if you find one makes your more productive, improves your concentration then use it!

Yes indeed.  So why are you concentrating your attention on something that is "immaterial"?

Quote
If you truly want to talk about programming languages then all these issues are part of the subject, so get used to it.

Programming languages are more about semantics and behaviour than they are about syntax. If you don't believe that, then consider the many very different things that "f.g(h)" can cause to happen in different languages.

So why are you concentrating your attention on syntax?
Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 09:17:42 am
And the whole idea of programming languages with different "reserved" words is pretty bad. It completely shuts down international teamwork. People don't do this not because it is impossible, but because it is impractical and nobody wants it.

I tend to agree.

How many languages are there? How many won't be supported?

In many of my jobs it has been normal to have several different nationalities in the team.
If multiple languages are allowed, will if( aRectangle.সূচনাকরুন() ) {...} be allowed? If not, why not?

Which language would be stored in the source code control system?
How would "diffs" be done?
Who will write the tool that allows reconciles code in Basque, Finnish, Malayalam, Arabic, Hebrew?
While we don't have any boustrophedonic languages any more, will the top-to-bottom languages be excluded?
And, of course there is the issue of pair programming? Which language wins out there?

How will IDEs cope when parsing code so they can provide ctrl-space style autocompletion?
Can a programmer add a method in Chinese to a class in Spanish with a superclass in Welsh?

Yup it is a whole can of worms. Makes APL's somewhat arbitrary symbols look like a sensible option.

Maybe there's a reason  the Algol experience hasn't been repeated!

APL's symbols weren't arbitrary read Iverson's Turing Award lecture, hopefully you'll learn something about this subject. Your confused questions make it clear you understand little about this subject.

Hence my use of the word "somewhat".

Nonetheless, to an extent all words in all languages are arbitrary. There is no reason why one specific group of animals should be called "cat" and another "dog" - and in many languages they aren't!

Quote
Everything I've said about this feature is OPTIONAL just like any number of coding standards or conventions. There is no computational reason for language keywords to be restricted to English, the abstract concepts are not English, or Japanese or Hebrew, they are universal  concepts just like mathematical ideas, or APL ideas.

Oh! You actually realise the points I made! From your arguments I didn't guess that.

Optionality is a problem. With C++ (and to a lesser extent with C) there are so many ways of doing the same thing, which leads to the requirement that one of many subsets of the language is chosen in each  project/company.

Unjustified complexity is a problem. Even the C++ committee designing the language didn't understand the consequences of the language features' interaction; in one case they had to have their noses rubbed in it!

Quote
You can use Google translate right now to translate Russian or Norwegian articles into English, is that a bad thing? does that cause nothing but problems and confusion? do people try to mix multiple languages within the same article? think about what I'm saying to you.

Being able to translate source code from English to Spanish to Dutch to Russian with absolutely no impact on the code's meaning or runtime behavior is actually a hugely powerful capability, one you clearly cannot perceive or grasp, it goes right over your head.

Next you'll be suggesting that ChatGPT produces output where it is guaranteed that the details are reliable.
Title: Re: A microcontroller programming language
Post by: Siwastaja on January 10, 2023, 10:38:25 am
And this is supposed to be used for microcontrollers

Don't forget, it was supposed to be specifically tailored for those professionals in the industry who write cycle-accurate code for PIC microcontrollers all day long!

The hilarity of this "Sherlock Holmes" character has totally surpassed anything I expected, although the signs were clear from the start. Excellent thread to follow!  :-DD
Title: Re: A microcontroller programming language
Post by: DiTBho on January 10, 2023, 11:04:09 am
And this is supposed to be used for microcontrollers
Don't forget, it was supposed to be specifically tailored for those professionals in the industry who write cycle-accurate code for PIC microcontrollers all day long!

The hilarity of this "Sherlock Holmes" character has totally surpassed anything I expected, although the signs were clear from the start. Excellent thread to follow!  :-DD

Indeed  :-DD
Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 11:30:08 am
And this is supposed to be used for microcontrollers

Don't forget, it was supposed to be specifically tailored for those professionals in the industry who write cycle-accurate code for PIC microcontrollers all day long!

Well, threads do drift and their titles do change :)

Quote
The hilarity of this "Sherlock Holmes" character has totally surpassed anything I expected, although the signs were clear from the start. Excellent thread to follow!  :-DD

It is quite fun, isn't it :)

Here's another of his fun threads: https://www.eevblog.com/forum/testgear/true-analog-scopes/msg4579102/#msg4579102 (https://www.eevblog.com/forum/testgear/true-analog-scopes/msg4579102/#msg4579102)
Title: Re: A microcontroller programming language
Post by: MIS42N on January 10, 2023, 11:45:46 am
And this is supposed to be used for microcontrollers

Don't forget, it was supposed to be specifically tailored for those professionals in the industry who write cycle-accurate code for PIC microcontrollers all day long!

That's me. The language is called assembler. Well, maybe not me as I don't consider myself a professional. But I do write cycle-accurate code (get the PIC to accurately detect a pulse arrival to the nearest 25ns while handling 9600 baud input in a soft UART and dealing with a 40kHz timer interrupt and a bit of I/O on the real UART on an 8-bit PIC without dropping the ball, we can do that).
Title: Re: A microcontroller programming language
Post by: DiTBho on January 10, 2023, 11:49:51 am
One famous book is Papadimitriou's Computational Complexity; a book that shows that theoretical computer science is practically a branch of mathematics, mostly concerned with studies of algorithms, so it also shows that even when people say "computer science" and they usually include many things which would not be considered mathematics, for instance computer architecture, specific programming languages, etc, well, certain aspects of both "sets" are shared { logic, proofs, ...}

So computer science is practically a branch of mathematics  :D


Title: Re: A microcontroller programming language
Post by: Nominal Animal on January 10, 2023, 12:09:00 pm
Instead of flat text files, the source code of this language would be better stored as a token stream, alongside with token-to-text mappings in each language for that project, say in a zip archive like Microsoft Office OOXML files.

Source code editors would handle lexical analysis, with the compiler, JIT, or interpreter processing the token stream instead.

Each token could be typographical (say, "Newline", "Indentation of one level", "Indentation of two levels"), syntactical (say, beginning of a quoted string, end of a quoted string, object member reference), operator (addition, subtraction, negation, multiplication, division, assignment, equality comparison), name, language keyword, and so on.  This would allow things like one developer seeing quoted strings as "Thus" whereas another would see it as «Thus» and yet another as “Thus”; assignment could be := or = or even equals, for each developer working on the same source, depending on their personal preferences (by just modifying their personal token mapping).  The editor would be responsible for ensuring whatever the developer writes is unambiguously tokenized.

Literal strings themselves could be included in the token mapping, although it probably should be independent of the developer token mapping, as it could be used for runtime localization (say, multi-language user interfaces).

One option for these mappings would be to store source as its Gödel number, assuming each token (and literal string) is assigned an unique natural number. 

For security, the ZIP files could be encrypted with a per-project key, a reverse public key.  (That is, the public key is used to encrypt the contents, and the private key used to decrypt the contents.)  Or you could use a centralized project key storage.  The latter would be very interesting in the business sense, as obviously the development of both the editing environment and the toolchain requires resources, so the vendor managing the project keys would ensure licensee validity, even while letting toolchain and IDE downloads be freely available.
Title: Re: A microcontroller programming language
Post by: JPortici on January 10, 2023, 12:13:48 pm
And this is supposed to be used for microcontrollers

Don't forget, it was supposed to be specifically tailored for those professionals in the industry who write cycle-accurate code for PIC microcontrollers all day long!

The hilarity of this "Sherlock Holmes" character has totally surpassed anything I expected, although the signs were clear from the start. Excellent thread to follow!  :-DD

I smelled something when "THERE SHOULD BE NO RESERVED WORDS" became "Not only there will be reserved words, but they will be translatable"
Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 12:47:00 pm
And this is supposed to be used for microcontrollers

Don't forget, it was supposed to be specifically tailored for those professionals in the industry who write cycle-accurate code for PIC microcontrollers all day long!

The hilarity of this "Sherlock Holmes" character has totally surpassed anything I expected, although the signs were clear from the start. Excellent thread to follow!  :-DD

I smelled something when "THERE SHOULD BE NO RESERVED WORDS" became "Not only there will be reserved words, but they will be translatable"

Despite replying to my post, he omitted to address the issue of how that would interact with tooling like IDEs and repositories.

Nor did he have anything to say about whether the translatable-non-keywords concept would or wouldn't be extemded to, say, if( aRectangle.সূচনাকরুন() ) {...} ? If not, why not?

And then there's concentrating on syntax, while resolutely avoiding the more important topic of what "f.g(h)" might actually mean/do in his language :)
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 01:36:04 pm
Those wishing to seriously discuss programming languages should at least make an effort to understand the difference between grammar and syntax. Its like trying to explain a passive tuned circuit to a class who don't know what DC resistance is.
Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 01:41:02 pm
Those wishing to seriously discuss programming languages should at least make an effort to understand the difference between grammar and syntax. Its like trying to explain a passive tuned circuit to a class who don't know what DC resistance is.

Ditto syntax and semantics - and especially their relative importance to the usefulness of the tool.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 01:42:39 pm
And this is supposed to be used for microcontrollers

Don't forget, it was supposed to be specifically tailored for those professionals in the industry who write cycle-accurate code for PIC microcontrollers all day long!

The hilarity of this "Sherlock Holmes" character has totally surpassed anything I expected, although the signs were clear from the start. Excellent thread to follow!  :-DD

I smelled something when "THERE SHOULD BE NO RESERVED WORDS" became "Not only there will be reserved words, but they will be translatable"

Despite replying to my post, he omitted to address the issue of how that would interact with tooling like IDEs and repositories.

Nor did he have anything to say about whether the translatable-non-keywords concept would or wouldn't be extemded to, say, if( aRectangle.সূচনাকরুন() ) {...} ? If not, why not?

And then there's concentrating on syntax, while resolutely avoiding the more important topic of what "f.g(h)" might actually mean/do in his language :)

It is you who is fixated on syntax, I rarely use the term here myself. There are many inaccuracies in your posts here, for example one can already, today write identifiers using Eastern characters, many programming languages do this, its been around for decades, nothing new.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 01:43:43 pm
Those wishing to seriously discuss programming languages should at least make an effort to understand the difference between grammar and syntax. Its like trying to explain a passive tuned circuit to a class who don't know what DC resistance is.

Ditto syntax and semantics - and especially their relative importance to the usefulness of the tool.

Are they the same thing?
Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 01:46:12 pm
And this is supposed to be used for microcontrollers

Don't forget, it was supposed to be specifically tailored for those professionals in the industry who write cycle-accurate code for PIC microcontrollers all day long!

The hilarity of this "Sherlock Holmes" character has totally surpassed anything I expected, although the signs were clear from the start. Excellent thread to follow!  :-DD

I smelled something when "THERE SHOULD BE NO RESERVED WORDS" became "Not only there will be reserved words, but they will be translatable"

Despite replying to my post, he omitted to address the issue of how that would interact with tooling like IDEs and repositories.

Nor did he have anything to say about whether the translatable-non-keywords concept would or wouldn't be extemded to, say, if( aRectangle.সূচনাকরুন() ) {...} ? If not, why not?

And then there's concentrating on syntax, while resolutely avoiding the more important topic of what "f.g(h)" might actually mean/do in his language :)

It is you who is fixated on syntax, I rarely use the term here myself. There are many inaccuracies in your posts here, for example one can already, today write identifiers using Eastern characters, many programming languages do this, its been around for decades, nothing new.

And still you fail to address important considerations that don't mesh with your chosen path and viewpoint!
Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 01:47:56 pm
Those wishing to seriously discuss programming languages should at least make an effort to understand the difference between grammar and syntax. Its like trying to explain a passive tuned circuit to a class who don't know what DC resistance is.

Ditto syntax and semantics - and especially their relative importance to the usefulness of the tool.

Are they the same thing?

That question is revealing. But probably not in the way you intend!
Title: Re: A microcontroller programming language
Post by: DiTBho on January 10, 2023, 01:57:52 pm
Note the arrogance.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 02:49:36 pm
The vocabulary rendered on screen is as immaterial as the colors or fonts one uses in their IDE, I don't care if you use bold purple for C keywords or italic blue, why would I care that you have that freedom? if you find one makes your more productive, improves your concentration then use it!

Yes indeed.  So why are you concentrating your attention on something that is "immaterial"?

You have absolutely no idea what I am concentrating my attention on. To make the language agnostic with respect to keyword vocabulary, takes design, design effort.

Quote
If you truly want to talk about programming languages then all these issues are part of the subject, so get used to it.

Programming languages are more about semantics and behaviour than they are about syntax. If you don't believe that, then consider the many very different things that "f.g(h)" can cause to happen in different languages.

So why are you concentrating your attention on syntax?

As I said I am designing a language more or less from scratch. One cannot design a language without designing a grammar and it seems you are unaware, semantics is part of grammar. Discussions about coroutines or set membership or interrupt handling or memory allocation/freeing are discussions about semantics and all of these have been discussed by me in this thread.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 02:52:12 pm
And the whole idea of programming languages with different "reserved" words is pretty bad. It completely shuts down international teamwork. People don't do this not because it is impossible, but because it is impractical and nobody wants it.

I tend to agree.

How many languages are there? How many won't be supported?

In many of my jobs it has been normal to have several different nationalities in the team.
If multiple languages are allowed, will if( aRectangle.সূচনাকরুন() ) {...} be allowed? If not, why not?

Which language would be stored in the source code control system?
How would "diffs" be done?
Who will write the tool that allows reconciles code in Basque, Finnish, Malayalam, Arabic, Hebrew?
While we don't have any boustrophedonic languages any more, will the top-to-bottom languages be excluded?
And, of course there is the issue of pair programming? Which language wins out there?

How will IDEs cope when parsing code so they can provide ctrl-space style autocompletion?
Can a programmer add a method in Chinese to a class in Spanish with a superclass in Welsh?

Yup it is a whole can of worms. Makes APL's somewhat arbitrary symbols look like a sensible option.

Maybe there's a reason  the Algol experience hasn't been repeated!

APL's symbols weren't arbitrary read Iverson's Turing Award lecture, hopefully you'll learn something about this subject. Your confused questions make it clear you understand little about this subject.

Hence my use of the word "somewhat".

Nonetheless, to an extent all words in all languages are arbitrary. There is no reason why one specific group of animals should be called "cat" and another "dog" - and in many languages they aren't!

Quote
Everything I've said about this feature is OPTIONAL just like any number of coding standards or conventions. There is no computational reason for language keywords to be restricted to English, the abstract concepts are not English, or Japanese or Hebrew, they are universal  concepts just like mathematical ideas, or APL ideas.

Oh! You actually realise the points I made! From your arguments I didn't guess that.

Why guess? use facts and logic instead, try it.

Optionality is a problem. With C++ (and to a lesser extent with C) there are so many ways of doing the same thing, which leads to the requirement that one of many subsets of the language is chosen in each  project/company.

Unjustified complexity is a problem. Even the C++ committee designing the language didn't understand the consequences of the language features' interaction; in one case they had to have their noses rubbed in it!

Regarding the many shortcoming of C and C++, you're preaching to the converted here.

Quote
You can use Google translate right now to translate Russian or Norwegian articles into English, is that a bad thing? does that cause nothing but problems and confusion? do people try to mix multiple languages within the same article? think about what I'm saying to you.

Being able to translate source code from English to Spanish to Dutch to Russian with absolutely no impact on the code's meaning or runtime behavior is actually a hugely powerful capability, one you clearly cannot perceive or grasp, it goes right over your head.

Next you'll be suggesting that ChatGPT produces output where it is guaranteed that the details are reliable.

Another guess I see.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 02:54:57 pm
One famous book is Papadimitriou's Computational Complexity; a book that shows that theoretical computer science is practically a branch of mathematics, mostly concerned with studies of algorithms, so it also shows that even when people say "computer science" and they usually include many things which would not be considered mathematics, for instance computer architecture, specific programming languages, etc, well, certain aspects of both "sets" are shared { logic, proofs, ...}

So computer science is practically a branch of mathematics  :D

Mathematics is not science, science is about discerning structure, patterns, symmetry in the natural world, computers are not "natural" they are created by mind.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 03:03:54 pm
Instead of flat text files, the source code of this language would be better stored as a token stream, alongside with token-to-text mappings in each language for that project, say in a zip archive like Microsoft Office OOXML files.

Source code editors would handle lexical analysis, with the compiler, JIT, or interpreter processing the token stream instead.

Each token could be typographical (say, "Newline", "Indentation of one level", "Indentation of two levels"), syntactical (say, beginning of a quoted string, end of a quoted string, object member reference), operator (addition, subtraction, negation, multiplication, division, assignment, equality comparison), name, language keyword, and so on.  This would allow things like one developer seeing quoted strings as "Thus" whereas another would see it as «Thus» and yet another as “Thus”; assignment could be := or = or even equals, for each developer working on the same source, depending on their personal preferences (by just modifying their personal token mapping).  The editor would be responsible for ensuring whatever the developer writes is unambiguously tokenized.

Literal strings themselves could be included in the token mapping, although it probably should be independent of the developer token mapping, as it could be used for runtime localization (say, multi-language user interfaces).

One option for these mappings would be to store source as its Gödel number, assuming each token (and literal string) is assigned an unique natural number. 

For security, the ZIP files could be encrypted with a per-project key, a reverse public key.  (That is, the public key is used to encrypt the contents, and the private key used to decrypt the contents.)  Or you could use a centralized project key storage.  The latter would be very interesting in the business sense, as obviously the development of both the editing environment and the toolchain requires resources, so the vendor managing the project keys would ensure licensee validity, even while letting toolchain and IDE downloads be freely available.

This is quite true, this has crossed my mind too. But then we would not be able to peruse raw text files, we'd need tooling to replace the abstract token codes with some real human vocabulary.

The ability to use multiple vocabularies is entirely optional, if someone wants to work wholly in English they can, there's no compulsion or even visibility to the feature.

C lets us write identifiers using the Greek alphabet if we want, I could write Ω for a variable representing resistance, I can do that now in C, C++ and umpteen languages, funny how the cavemen here are not getting all bent out of shape by this optional feature.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 03:09:31 pm
And this is supposed to be used for microcontrollers

Don't forget, it was supposed to be specifically tailored for those professionals in the industry who write cycle-accurate code for PIC microcontrollers all day long!

The hilarity of this "Sherlock Holmes" character has totally surpassed anything I expected, although the signs were clear from the start. Excellent thread to follow!  :-DD

I smelled something when "THERE SHOULD BE NO RESERVED WORDS" became "Not only there will be reserved words, but they will be translatable"

Despite replying to my post, he omitted to address the issue of how that would interact with tooling like IDEs and repositories.

Nor did he have anything to say about whether the translatable-non-keywords concept would or wouldn't be extemded to, say, if( aRectangle.সূচনাকরুন() ) {...} ? If not, why not?

And then there's concentrating on syntax, while resolutely avoiding the more important topic of what "f.g(h)" might actually mean/do in his language :)

Very well, I'll indulge your little questions:

1. IDE's? It is possible to imbue an IDE or editor with an ability to instantly re-render source code in any vocabulary, what more do you need to know?
2. Repositories? Well if that's a struggle for you I can't see why. For any open source project one could impose the rule that all pull-requests are converted to (say) English when merged, only English source gets merged, that's a basic team decision, not a problem.
3. One can today mix character sets in C code, I can name a struct "parameters" and give it an int member named "Ω", languages are flexible, use them as you wish, why are you asking me how you'd choose to name things?


Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 03:10:37 pm
And this is supposed to be used for microcontrollers

Don't forget, it was supposed to be specifically tailored for those professionals in the industry who write cycle-accurate code for PIC microcontrollers all day long!

The hilarity of this "Sherlock Holmes" character has totally surpassed anything I expected, although the signs were clear from the start. Excellent thread to follow!  :-DD

I smelled something when "THERE SHOULD BE NO RESERVED WORDS" became "Not only there will be reserved words, but they will be translatable"

Despite replying to my post, he omitted to address the issue of how that would interact with tooling like IDEs and repositories.

Nor did he have anything to say about whether the translatable-non-keywords concept would or wouldn't be extemded to, say, if( aRectangle.সূচনাকরুন() ) {...} ? If not, why not?

And then there's concentrating on syntax, while resolutely avoiding the more important topic of what "f.g(h)" might actually mean/do in his language :)

It is you who is fixated on syntax, I rarely use the term here myself. There are many inaccuracies in your posts here, for example one can already, today write identifiers using Eastern characters, many programming languages do this, its been around for decades, nothing new.

And still you fail to address important considerations that don't mesh with your chosen path and viewpoint!

I disagree.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 03:12:11 pm
Those wishing to seriously discuss programming languages should at least make an effort to understand the difference between grammar and syntax. Its like trying to explain a passive tuned circuit to a class who don't know what DC resistance is.

Ditto syntax and semantics - and especially their relative importance to the usefulness of the tool.

Are they the same thing?

That question is revealing. But probably not in the way you intend!

That's not an answer to the question, do "grammar" and "syntax" mean the same thing when discussing linguistics? do you know the answer? if you do why the reticence in honestly answering it?
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 03:28:09 pm
I'd like to ask, please, if you have nothing to contribute other than invective, ridicule and sarcasm, then you consider not posting to the thread?

I expect adults to discuss like adults not like juveniles, too many posts here are non-constructive, little more than name calling, personal expressions of distaste, of outrage and blatant pettiness.

This is sadly a side-effect of people hiding behind the internet, insulting those with whom they disagree rather than politely disagreeing, I have a policy that I never address a person online in a way I would not address them in person in a meeting of peers, only the coward behaves differently in person and online.

If the subject doesn't interest you, if you are a know-all with a closed mind or simply if you think I'm an idiot, a dunce, a bozo, that's fine - just don't post here, please...

Title: Re: A microcontroller programming language
Post by: Nominal Animal on January 10, 2023, 05:03:28 pm
Instead of flat text files, the source code of this language would be better stored as a token stream, alongside with token-to-text mappings in each language for that project, say in a zip archive like Microsoft Office OOXML files.

Source code editors would handle lexical analysis, with the compiler, JIT, or interpreter processing the token stream instead.

Each token could be typographical (say, "Newline", "Indentation of one level", "Indentation of two levels"), syntactical (say, beginning of a quoted string, end of a quoted string, object member reference), operator (addition, subtraction, negation, multiplication, division, assignment, equality comparison), name, language keyword, and so on.  This would allow things like one developer seeing quoted strings as "Thus" whereas another would see it as «Thus» and yet another as “Thus”; assignment could be := or = or even equals, for each developer working on the same source, depending on their personal preferences (by just modifying their personal token mapping).  The editor would be responsible for ensuring whatever the developer writes is unambiguously tokenized.

Literal strings themselves could be included in the token mapping, although it probably should be independent of the developer token mapping, as it could be used for runtime localization (say, multi-language user interfaces).

One option for these mappings would be to store source as its Gödel number, assuming each token (and literal string) is assigned an unique natural number. 

For security, the ZIP files could be encrypted with a per-project key, a reverse public key.  (That is, the public key is used to encrypt the contents, and the private key used to decrypt the contents.)  Or you could use a centralized project key storage.  The latter would be very interesting in the business sense, as obviously the development of both the editing environment and the toolchain requires resources, so the vendor managing the project keys would ensure licensee validity, even while letting toolchain and IDE downloads be freely available.

This is quite true, this has crossed my mind too. But then we would not be able to peruse raw text files, we'd need tooling to replace the abstract token codes with some real human vocabulary.

How else would multicultural and multilingual development teams cooperate?

Consider this the programming equivalent of personal pronouns: each subgroup gets to define how the code looks to them, without any oppressor forcing them to use a specific form.  I'm quite sure this is the future of socially aware software development.

(This might also open up interesting possibilities for funding the development of such a programming language.)
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 05:48:35 pm
Instead of flat text files, the source code of this language would be better stored as a token stream, alongside with token-to-text mappings in each language for that project, say in a zip archive like Microsoft Office OOXML files.

Source code editors would handle lexical analysis, with the compiler, JIT, or interpreter processing the token stream instead.

Each token could be typographical (say, "Newline", "Indentation of one level", "Indentation of two levels"), syntactical (say, beginning of a quoted string, end of a quoted string, object member reference), operator (addition, subtraction, negation, multiplication, division, assignment, equality comparison), name, language keyword, and so on.  This would allow things like one developer seeing quoted strings as "Thus" whereas another would see it as «Thus» and yet another as “Thus”; assignment could be := or = or even equals, for each developer working on the same source, depending on their personal preferences (by just modifying their personal token mapping).  The editor would be responsible for ensuring whatever the developer writes is unambiguously tokenized.

Literal strings themselves could be included in the token mapping, although it probably should be independent of the developer token mapping, as it could be used for runtime localization (say, multi-language user interfaces).

One option for these mappings would be to store source as its Gödel number, assuming each token (and literal string) is assigned an unique natural number. 

For security, the ZIP files could be encrypted with a per-project key, a reverse public key.  (That is, the public key is used to encrypt the contents, and the private key used to decrypt the contents.)  Or you could use a centralized project key storage.  The latter would be very interesting in the business sense, as obviously the development of both the editing environment and the toolchain requires resources, so the vendor managing the project keys would ensure licensee validity, even while letting toolchain and IDE downloads be freely available.

This is quite true, this has crossed my mind too. But then we would not be able to peruse raw text files, we'd need tooling to replace the abstract token codes with some real human vocabulary.

How else would multicultural and multilingual development teams cooperate?

Consider this the programming equivalent of personal pronouns: each subgroup gets to define how the code looks to them, without any oppressor forcing them to use a specific form.  I'm quite sure this is the future of socially aware software development.

(This might also open up interesting possibilities for funding the development of such a programming language.)

Yes, good questions and I don't know exactly how a team might choose to work, let me elaborate on what I've been doing then try to answer you.

At this stage I've proven that a self-consistent non-ambiguous grammar can be devised that is insensitive to the exact spelling of its keywords, I wasn't 100% sure but suspected this was possible for a grammar that has no reserved words (like PL/I, Fortran etc.). I also wasn't sure if current parser tools were able to do what I needed, I've used them before and they can block progress sometimes once some idea is tried in a grammar, also hand crafted lexers and parsers are possible (I've written these before) but they make experiments, changes very hard, very slow going, so slow that one ends up not experimenting much.

Anyway Antlr is beyond my highest expectations, it is truly very powerful and far beyond anything I could craft by hand, in a couple of seconds I can tweak a grammar rule and regen the parser source code and test it, very powerful indeed.

At parse time I specify the keyword lexicon code "en" (English), "fr" (French) etc and the generated lexer knows what the keywords should be for that language code, the parser code is agnostic, has no knowledge of keyword spelling at all other than in the abstract sense.

So, how to use this?

Well the language code could be a compiler option or a preprocessor setting within a source file or inferred from the name of the file, there are several ways one could do that, not a huge effort.

Also the parser code is a class library (Java or C#), so can be used as part of a compiler or within another tool, for example I'm looking at a simple command line tool that can consume a source file in one language and create an output in another, this is not very hard to do, yes there's code involved to recreate a text file from the parse tree, spaces, comments, line endings etc but that is - in principle - not much of a problem, I asked the Antlr team, I've not looked at that in earnest yet but I might do soon.

Anyway it should be easy to build such power into an IDE or editor, there are numerous editors that support all kinds of extensibility (VS Code is very good) so one could just open and edit a *.ipl" file and click a dropdown "Convert to..." where we can choose some target and the tool will instantly refresh the file with the source code in that chosen language.

Or just as easily, we can envisage "Save As..." where we can save "test_1_abc.ipl" as (say) "test_1_abc.ru.ipl" (or any name really) and specify "Russian" as the choice of keyword lexicon, the tool would regen that file being saved into the equivalent in Russian using the same tool I describe above.

Or "Open As" could open a file in whatever language, into whatever language one wanted to edit the file in!

I've been thinking about a way to detect the lexicon used in a source file, there are ways to do this.

These ideas mean we could live in a world where we can open any file in any language yet see it only in our chosen language and when we save it it gets saved back in its original language, all invisibly, these are all serious possibilities.

These and other reasons are why I've been focused on the grammar, one cannot start to implement semantic processing (and code generation really) until one commits to a grammar and once you've written that stuff it is very very very hard indeed to go back and adjust the grammar without often complex rework on the semantic processor etc, the goal has been to get to a grammar that can support essential features and then start to design the semantic processing and then the rest of the parts in a compiler's back end.

The back end is to all intents and purposes a relatively routine phase of a project like this, back end's are decoupled (largely) from the front end grammar but we do need the middle - the semantic phase, optimizers etc.

Antlr does not generate (or assist with the generation of) an abstract syntax tree, that code is an essential part of a full compiler and as you can appreciate the AST generator must consume the parse tree so if the grammar had to change much, the parse tree would and then we'd have to rework the AST generatror along with any code we'd written that was consuming that AST, lots of needless work, wasted time.

These and other real world issues are being glossed over by some of the naïve detractors posting in this thread, they asked several times "why are you fixated on syntax rather than the nitty gritty compiler and code generator" I'm afraid such a question only reveals their naivety about real world "gloves off" compiler design.

Just to stress, the multiple keyword lexicon was not initially on my list of goals for the grammar, I only added it after realizing it was possible and with very little effort when using powerful tooling like Antlr. An eventual compiler would let one work wholly in a single language if that's all they wanted to do, seamlessly, there's no impact on simple basic use by having this multi language feature, if one doesn't care for it then disregard it.
Title: Re: A microcontroller programming language
Post by: JPortici on January 10, 2023, 06:15:23 pm
How else would multicultural and multilingual development teams cooperate?

Consider this the programming equivalent of personal pronouns: each subgroup gets to define how the code looks to them, without any oppressor forcing them to use a specific form.  I'm quite sure this is the future of socially aware software development.

(This might also open up interesting possibilities for funding the development of such a programming language.)

I can't honestly tell if you are being serious right now.
To me that sounds like a nightmare. Simple syntax, having to use a specific language in a project instead of each one their own is a compromize we all have to make to share a common base. And that's good. What's good engineering if not the best possible compromize?

My PC has some settings in italian locale, other in english locale. Some software decide to use one, other software the other, without consistency. And there were cases in which the locale were mixed, who knows why.
Since you bring up excel, I also have had (very few) documents that refued to work because some functions wouldn't be translated from one language to the other. In OO Calc all menus are in english because system language is english but i have to use italian formulae. Go figure.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 06:21:48 pm
How else would multicultural and multilingual development teams cooperate?

Consider this the programming equivalent of personal pronouns: each subgroup gets to define how the code looks to them, without any oppressor forcing them to use a specific form.  I'm quite sure this is the future of socially aware software development.

(This might also open up interesting possibilities for funding the development of such a programming language.)

I can't honestly tell if you are being serious right now.
To me that sounds like a nightmare. Simple syntax, having to use a specific language in a project instead of each one their own is a compromize we all have to make to share a common base. And that's good. What's good engineering if not the best possible compromize?

My PC has some settings in italian locale, other in english locale. Some software decide to use one, other software the other, without consistency. And there were cases in which the locale were mixed, who knows why.
Since you bring up excel, I also have had (very few) documents that refued to work because some functions wouldn't be translated from one language to the other. In OO Calc all menus are in english because system language is english but i have to use italian formulae. Go figure.

I don't see any advantage in a programming language that recognizes only English keywords over one that can recognize English and - optionally - other cultures' keywords too. What advantage does such a programming language offer and to whom? what can the former do that the latter cannot?



Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 06:23:59 pm
How else would multicultural and multilingual development teams cooperate?

Consider this the programming equivalent of personal pronouns: each subgroup gets to define how the code looks to them, without any oppressor forcing them to use a specific form.  I'm quite sure this is the future of socially aware software development.

(This might also open up interesting possibilities for funding the development of such a programming language.)

I can't honestly tell if you are being serious right now.
To me that sounds like a nightmare. Simple syntax, having to use a specific language in a project instead of each one their own is a compromize we all have to make to share a common base. And that's good. What's good engineering if not the best possible compromize?

I'm 99.999% sure he isn't being serious, partly because he (almost always  >:D ) makes sense :)

Instead he is one of several people pointing out to the OP that, while his ideas might be cute in some way, in practice they will be a can of worms. As of a couple of posts ago, the OP appears to accept that he will also be rewriting the development tools around the language!
Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 06:26:41 pm
I don't see any advantage in a programming language that recognizes only English keywords over one that can recognize English and - optionally - other cultures' keywords too. What advantage does such a programming language offer and to whom? what can the former do that the latter cannot?

Most people will ask a different question, along the lines of "What is the benefit of a language that has (non-) keywords in many languages? How does it improve the code quality and reduce cost?

In other words, don't confuse "features" with "benefits".
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 06:28:08 pm
How else would multicultural and multilingual development teams cooperate?

Consider this the programming equivalent of personal pronouns: each subgroup gets to define how the code looks to them, without any oppressor forcing them to use a specific form.  I'm quite sure this is the future of socially aware software development.

(This might also open up interesting possibilities for funding the development of such a programming language.)

I can't honestly tell if you are being serious right now.
To me that sounds like a nightmare. Simple syntax, having to use a specific language in a project instead of each one their own is a compromize we all have to make to share a common base. And that's good. What's good engineering if not the best possible compromize?

I'm 99.999% sure he isn't being serious, partly because he (almost always  >:D ) makes sense :)

Instead he is one of several people pointing out to the OP that, while his ideas might be cute in some way, in practice they will be a can of worms. As of a couple of posts ago, the OP appears to accept that he will also be rewriting the development tools around the language!

You must be paraphrasing me, can you quote the post where I said "I will also be rewriting the development tools around the language"? I said no such thing, nor would I ever express myself in such a sloppy, cavalier manner.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 06:29:46 pm
I don't see any advantage in a programming language that recognizes only English keywords over one that can recognize English and - optionally - other cultures' keywords too. What advantage does such a programming language offer and to whom? what can the former do that the latter cannot?

Most people will ask a different question, along the lines of "What is the benefit of a language that has (non-) keywords in many languages? How does it improve the code quality and reduce cost?

In other words, don't confuse "features" with "benefits".

Call me old fashioned but I like see questions answered in the same order they are asked, if you'd prefer to evade it then do so, but it doesn't do much for your speculative claims.

So, lets try again: What advantage does such a programming language offer and to whom? what can the former do that the latter cannot? perhaps you're at a loss for words? if so I'm forever grateful.

Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 08:11:32 pm
How else would multicultural and multilingual development teams cooperate?

Consider this the programming equivalent of personal pronouns: each subgroup gets to define how the code looks to them, without any oppressor forcing them to use a specific form.  I'm quite sure this is the future of socially aware software development.

(This might also open up interesting possibilities for funding the development of such a programming language.)

I can't honestly tell if you are being serious right now.
To me that sounds like a nightmare. Simple syntax, having to use a specific language in a project instead of each one their own is a compromize we all have to make to share a common base. And that's good. What's good engineering if not the best possible compromize?

I'm 99.999% sure he isn't being serious, partly because he (almost always  >:D ) makes sense :)

Instead he is one of several people pointing out to the OP that, while his ideas might be cute in some way, in practice they will be a can of worms. As of a couple of posts ago, the OP appears to accept that he will also be rewriting the development tools around the language!

You must be paraphrasing me, can you quote the post where I said "I will also be rewriting the development tools around the language"? I said no such thing, nor would I ever express myself in such a sloppy, cavalier manner.

It is implicit in https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4629565/#msg4629565 (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4629565/#msg4629565)
Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 08:16:34 pm
I don't see any advantage in a programming language that recognizes only English keywords over one that can recognize English and - optionally - other cultures' keywords too. What advantage does such a programming language offer and to whom? what can the former do that the latter cannot?

Most people will ask a different question, along the lines of "What is the benefit of a language that has (non-) keywords in many languages? How does it improve the code quality and reduce cost?

In other words, don't confuse "features" with "benefits".

Call me old fashioned but I like see questions answered in the same order they are asked, if you'd prefer to evade it then do so, but it doesn't do much for your speculative claims.

So, lets try again: What advantage does such a programming language offer and to whom? what can the former do that the latter cannot? perhaps you're at a loss for words? if so I'm forever grateful.

Call me old fashioned, but I like to skip over unimportant questions and concentrate on the "touchstone" questions, i.e. those where the answer indicates which direction to take.

Fundamentally, answering questions is relatively easy. It is more difficult (and rewarding) to find the right question to answer.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 08:20:26 pm
How else would multicultural and multilingual development teams cooperate?

Consider this the programming equivalent of personal pronouns: each subgroup gets to define how the code looks to them, without any oppressor forcing them to use a specific form.  I'm quite sure this is the future of socially aware software development.

(This might also open up interesting possibilities for funding the development of such a programming language.)

I can't honestly tell if you are being serious right now.
To me that sounds like a nightmare. Simple syntax, having to use a specific language in a project instead of each one their own is a compromize we all have to make to share a common base. And that's good. What's good engineering if not the best possible compromize?

I'm 99.999% sure he isn't being serious, partly because he (almost always  >:D ) makes sense :)

Instead he is one of several people pointing out to the OP that, while his ideas might be cute in some way, in practice they will be a can of worms. As of a couple of posts ago, the OP appears to accept that he will also be rewriting the development tools around the language!

You must be paraphrasing me, can you quote the post where I said "I will also be rewriting the development tools around the language"? I said no such thing, nor would I ever express myself in such a sloppy, cavalier manner.

It is implicit in https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4629565/#msg4629565 (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4629565/#msg4629565)

No it isn't, you're imagining things.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 08:22:38 pm
I don't see any advantage in a programming language that recognizes only English keywords over one that can recognize English and - optionally - other cultures' keywords too. What advantage does such a programming language offer and to whom? what can the former do that the latter cannot?

Most people will ask a different question, along the lines of "What is the benefit of a language that has (non-) keywords in many languages? How does it improve the code quality and reduce cost?

In other words, don't confuse "features" with "benefits".

Call me old fashioned but I like see questions answered in the same order they are asked, if you'd prefer to evade it then do so, but it doesn't do much for your speculative claims.

So, lets try again: What advantage does such a programming language offer and to whom? what can the former do that the latter cannot? perhaps you're at a loss for words? if so I'm forever grateful.

Call me old fashioned, but I like to skip over unimportant questions and concentrate on the "touchstone" questions, i.e. those where the answer indicates which direction to take.

I get it, and an "unimportant question" is defined as "a question I do not want to answer because it will make me look stupid if I do answer it honestly", yes?

Fundamentally, answering questions is relatively easy. It is more difficult (and rewarding) to find the right question to answer.

(or right question to evade!)

I'm just glad you're not designing a language, with "logic" like that it'd be more confusing than obfuscated C.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 09:33:45 pm
This thread is incomplete, until the new programming language name, is written in a significant number of the thousands of languages of this world.

7,117 languages:
https://www.theintrepidguide.com/how-many-languages-are-there-in-the-world/ (https://www.theintrepidguide.com/how-many-languages-are-there-in-the-world/)

Ideally, including Klingon and in pure mathematics.

The names, will be interesting in the languages which don't have a written form (if applicable).

Ideally express this list, in the form of an example program, written in your new language.

Let's say, that that program is 10,000 lines of source.

Then demonstrate how easy it is to translate that 10,000 (very approximately) line example code, into the required, 1,000 supported foreign languages.

N.B. Make sure that the program is very well documented, to put on a good show.  But also, don't forget to translate all those 1,000 example programs (including all the comments! but also the variable/constant/function/procedure/etc names), into their respective native foreign language.

It can't take very long to do that task, because if it did, you might be illustrating a point (via sarcasm), that the OP may have made one or two, slight design errors in the design of this proposed new language.

For bonus points.  Please state how many different keyboards were needed, to type the various symbols/characters in, and also explain how you coped, with any languages that google translate (or similar), doesn't currently handle.

Adding a keyword lexicon is a simple matter of editing a JSON file and creating a pull request, I don't care how many distinct languages they get created for, the point is they, can be, if somebody want's one they are welcome to add it.

I don't care about keyboards or input methods, anyone who uses their native alphabet today already know how to use their native alphabet I suspect. If they do not want to or find it difficult then they won't use it, if they find it easy they will use it - why is this hard for you to grasp?

You have a rare gift for making the simple appear complex, no wonder you struggle so much with some of this.

You bring to mind Sir Walter Raleigh:

https://youtu.be/n-Zm8H6G108?t=58 (https://youtu.be/n-Zm8H6G108?t=58)




Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 10:09:20 pm
Adding a keyword lexicon is a simple matter of editing a JSON file and creating a pull request, I don't care how many distinct languages they get created for, the point is they, can be, if somebody want's one they are welcome to add it.

I don't care about keyboards or input methods, anyone who uses their native alphabet today already know how to use their native alphabet I suspect. If they do not want to or find it difficult then they won't use it, if they find it easy they will use it - why is this hard for you to grasp?

You have a rare gift for making the simple appear complex, no wonder you struggle so much with some of this.

Great, since it is so simple, and my first attempt at getting you to see sense, doesn't seem to have worked...
Il était une fois dans un pays loin, loin,
Es war einmal in einem Land weit, weit entfernt,
Había una vez en un país lejano, lejano,
story = "Once upon a time in a land far, far away, there lived a little dragon with scales of red and gray. He breathed fire and flew through the skies, fierce and mighty, with his head held high."
print(story)
(setf story (concatenate 'string "Once upon a time in a land far, far away, there lived a little dragon with scales of red and gray. He breathed fire and flew through the skies, fierce and mighty, with his head held high."))
(print story)
遠くの遠い国に、昔の話があった。
赤と灰色の鱗を持った小さなドラゴンが住んでいた。
彼は火を吹いて空を飛んでいた、
強くて威風堂々と頭を高く持っていた。
C'era una volta in un paese lontano, lontano,
很久很久以前,在一个遥远的国度里
Жил-был на свете дракон маленький,

كان يوما في بلد يبعد بعيدا،
كان هناك تنين صغير ذو تشعة من الأحمر والرمادي.
وقال انه يتنفس النار ويطير من خلال السماوات،
قوي وشجاع، مع رأسه العالي
nuqneH 'ej nuqDaq vIlo'laHbe'chugh yImev
Daq ghaH Daq vIneHmeH Daqmey tIqwIj
Daq vISam Daqmey Daq vIlegh
DaqmeyDaq vIneHmeH Daq vIbe'
Havia uma vez, em um país muito, muito longe,
Det var en gång i ett land långt, långt bort,
Er was eens in een land ver, ver weg,
Estis iam en lando for, for,
Kerran, kaukana kaukana maassa,
++++++++[>++++[>++>+++>+++>+<<<<-]>+<<<<-]>++.
(=<`#9]~6ZY32Vx/4Rs+0No-&Jk)8Cj
(space) (tab) (newline) (space) (space) (tab) (newline) (tab) (tab) (newline) (tab) (space) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline) (tab) (tab) (newline)

An improvement, definitely an improvement, this might be the most clearly written post I've seen from you for some time.

Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 10:18:24 pm
Just in case my last post is NOT obvious enough.

Programming projects, which have got lots of different languages (spoken) in them.  Can become nightmares to understand, maintain and process (for humans).  Therefore, great care, should be taken, before allowing/encouraging such features, in new programming languages.

In other words.  There can be extremely GOOD reasons, why existing programming languages, are the way, they are.

Don't forget the non-verbal non-written languages!

In this case I think one such phrase may be particularly relevant: <pheoooow, whooosh>. That can be loosely translated into English as "the sound made as the point whizzes past the OP".
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 10:22:39 pm
Just in case my last post is NOT obvious enough.

Programming projects, which have got lots of different languages (spoken) in them.  Can become nightmares to understand, maintain and process (for humans).  Therefore, great care, should be taken, before allowing/encouraging such features, in new programming languages.

First I have not advocated writing code in "different languages". I've enabled code to be written with a choice of keyword lexicons, applicable to an entire source file or project as decided by those using the language.

Second I would never and have not advocated not taking care when designing something.

Third code written in any programming language (e.g. C or C++) can "become a nightmare to understand, maintain and process (for humans)" so that complaint seems rather vacuous.

Fourth - by design - the language makes it simple to convert source code using one keyword lexicon into code that uses any of the other lexicons, that code will compile to an identical output and will behave identically at runtime.

Fifth since there are no programming languages that do what I've enabled this one to do, there is no evidence that it can lead to any of the world shattering cataclysms you seem to be terrified of, it is all in your own mind, a demon of your own making.
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 10:57:02 pm
In other words.  There can be extremely GOOD reasons, why existing programming languages, are the way, they are.

I'm sure that there are good reasons for why things were designed they way they were, no argument from me.

But something being designed without some feature does not prove that it was designed to not have that feature, that not having the feature was a design objective!

C was not designed to do OO so on that feeble basis you'd argue that OO programming language were a terrible idea? seriously?

This is the way of the Luddite, fear of change, demonizing innovation, fear of things foreign, sorry man I left that kind of provincial small world thinking behind long ago when I left the UK.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 11:05:00 pm
Just in case my last post is NOT obvious enough.

Programming projects, which have got lots of different languages (spoken) in them.  Can become nightmares to understand, maintain and process (for humans).  Therefore, great care, should be taken, before allowing/encouraging such features, in new programming languages.

In other words.  There can be extremely GOOD reasons, why existing programming languages, are the way, they are.

Don't forget the non-verbal non-written languages!

In this case I think one such phrase may be particularly relevant: <pheoooow, whooosh>. That can be loosely translated into English as "the sound made as the point whizzes past the OP".

That's it, you're ignored, nothing but vacuities and juvenile pettiness, later dude.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 11:12:04 pm
Just in case my last post is NOT obvious enough.

Programming projects, which have got lots of different languages (spoken) in them.  Can become nightmares to understand, maintain and process (for humans).  Therefore, great care, should be taken, before allowing/encouraging such features, in new programming languages.

In other words.  There can be extremely GOOD reasons, why existing programming languages, are the way, they are.

That's it, you're ignored, nothing but vacuities and juvenile pettiness, later dude.

Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 11:16:54 pm
Beautiful, that noise filter is sure nice, now back to business...
Title: Re: A microcontroller programming language
Post by: Sherlock Holmes on January 10, 2023, 11:27:06 pm
I'm very interested next (now that support for multiple keyword lexicons is working), in the subject of preprocessing. Some preprocessors like that found in PL/I were in essence a programming language (https://en.wikipedia.org/wiki/PL/I_preprocessor) for manipulating the source code, a true language with loops and if/then/else even preprocessor procedures.

Having read this recently about the language Zig (https://kevinlynagh.com/rust-zig/), I want to take a good long hard look at this to see how this can be leveraged for a new language.

Any thoughts, ideas or suggestions about this aspect of programming languages is welcome.

Title: Re: A microcontroller programming language
Post by: tggzzz on January 10, 2023, 11:51:04 pm
:palm:

Well at least I tried.

You We can lead a horse to water, but you we can't make it drink :)
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 11, 2023, 12:21:25 am
I think he is just trolling us with nothing but poor amount of repeated bullshit.

He does not event really want to discuss: nobody here wants that damn Babel feature, but he insists with nothing but aesthetic poor reasoning.

So, guys, give yourself a favor and give him nothing but a full ignore.
He will stop bothering us.

Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 11, 2023, 12:45:32 am
Aaaand... The OP has changed the thread name for a second time. I don't remember seeing that before!

What started out as "A new hardware oriented prorgramming language" has become completely boring: Yet Another Software Weenie's Me-too Compiler Exercise.

Changing the thread name is, IMHO, in bad taste. Good taste would have been to start a new thread on "My Own Programming Language: Imperium".
Title: Re: The Imperium programming language - IPL
Post by: retiredfeline on January 11, 2023, 03:52:01 am
I'm surprised OP did not call their language Elementary.  :-DD
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 11, 2023, 10:02:27 am
Allowing different language keywords will break a havoc in developing and maintaining tools which work at the source code level. Just do not do it.
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 11, 2023, 10:13:00 am
Luckily Linux kernel is written in C which doesn't allow using multilanguage keywords, and any preprocessor trickery in that regard is also prohibited. It would be a nightmare trying to read and maintain source code written using local language keywords, and the tools which should be able to support all current and future languages to be added at the later time. Klingon, anyone? I will repeat myself here: Just do not do it.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 11, 2023, 01:39:22 pm
I think he is just trolling us with nothing but poor amount of repeated bullshit.

He does not event really want to discuss: nobody here wants that damn Babel feature, but he insists with nothing but aesthetic poor reasoning.

So, guys, give yourself a favor and give him nothing but a full ignore.
He will stop bothering us.

If the thread offend you, go elsewhere, you too are engaging in harassment, you're ignored. Its frankly obscene to accuse me too of "bothering" anybody, a perversion of the facts.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 11, 2023, 01:54:08 pm
Allowing different language keywords will break a havoc in developing and maintaining tools which work at the source code level. Just do not do it.

Why? first only a single lexicon's keywords can be used in any file, it is not permitted to ever mix lexicons in a source file. The lexicon is specified in a compiler directive or can be overridden as a compiler option. Every source file uses one and only one lexicon and the default is English, if no directive or compile option is specified.

One must "opt in" to use other lexicons.

Any source file can be converted from any lexicon to any other using a simple utility (and class library by tool builders).

Owners of repositories can establish a check that all pull requests contain source in some defined lexicon, this is trivial to do with GitHub actions, every contributor would have to save their code with that lexicon.

A developer could work locally by seeing Bulgarian keywords but always have their files saved as English.

The analogy with fonts and colors in an IDE is good, nobody cares about how the source appears to a developer, all we care about is what is shared.



Title: Re: The Imperium programming language - IPL
Post by: YurkshireLad on January 11, 2023, 02:30:39 pm
I think he is just trolling us with nothing but poor amount of repeated bullshit.

He does not event really want to discuss, nobody wants that Babel feature, but he insists.

So, guys, give yourself a favor and give him nothing but a full ignore.
He will stop bothering us with his personal bullshit.

That makes sense (possible trolling).  Either that, or possibly significant communications difficulties, causing them to misunderstand too much, for them to make sense of it.  But there are many, possible explanations, of how this thread has come to exist.

Anyway, whatever the reason(s) is.  Something funny seems to be going on, even if it is not 100% clear, what.

Also, if someone wants to invent a brand new programming language.  Just go ahead and do it.  Don't spend all day, creating thread(s) about it.  Otherwise, it could end up going nowhere, fast.

Create a github project and start adding stuff and example code. We can all look at the example code and provide comments, except I can only understand English.
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 11, 2023, 03:03:31 pm
Allowing different language keywords will break a havoc in developing and maintaining tools which work at the source code level. Just do not do it.

Why? first only a single lexicon's keywords can be used in any file, it is not permitted to ever mix lexicons in a source file. The lexicon is specified in a compiler directive or can be overridden as a compiler option. Every source file uses one and only one lexicon and the default is English, if no directive or compile option is specified.

One must "opt in" to use other lexicons.

Any source file can be converted from any lexicon to any other using a simple utility (and class library by tool builders).

Owners of repositories can establish a check that all pull requests contain source in some defined lexicon, this is trivial to do with GitHub actions, every contributor would have to save their code with that lexicon.

A developer could work locally by seeing Bulgarian keywords but always have their files saved as English.

The analogy with fonts and colors in an IDE is good, nobody cares about how the source appears to a developer, all we care about is what is shared.

It is your language and project, so you can do whatever you want to. I have just expressed my opinion and concern here.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 11, 2023, 03:11:18 pm
A NOTE ON MULTIPLE KEYWORD LEXICONS

This is to clarify and define what this actually amounts too, there seems to be some misunderstandings about this so I'll clear it up once and for all.

The grammar - by design - does not reserve keywords, it allows an identifier to be a keyword. It does this not to facilitate obfuscation but to guarantee that a newly added keyword in a future version of the language will always be 100% backward compatible, that's one of the language's design goals, even if a newly added keyword has been used as an identifier in earlier source code by some developer, the code will compile fine, guaranteed.

During testing and conversation elsewhere outside of this forum, I realized that this absence of reserved words meant one could also select the keywords on a per-human-language basis and because no keyword is reserved the code would also compile fine, even if a developer had unwittingly used a keyword from some other language in their code. For example if an English developer had used "retour" as a variable name or function name then the fact that that is the French keyword for "return" would not matter even if we'd written the code using French keywords.

So the ability to be agnostic to the spelling of the keywords already, in and of itself, meant we could use keywords in any language we wanted to (if the grammar was updated to allow it) and the code would always compile, guaranteed, 100%.

The ability to use any cultures keyword spelling with zero impact on the generated code was already there, just "waiting in the wings". So I expanded the grammar (and therefore the lexer and parser) and created a proof of concept and it worked wonderfully.

So I decided to support this but make it "opt in". The system defaults to English, it behaves like any English based programming language. But if one wants to write code with some other language's keywords one can, all they need do is add a compiler directive to the source file (%lexicon("fr"); for example) and use the French keywords. Or one can specify the lexicon at compile time. Unless one does this the language behaves like a conventional English based language.

A source file can only use a single lexicon, one cannot mix French/Hungarian/English for example within a single source file, the lexicon language applies to an entire source file.

Because the compiler tools and libraries are reusable, it is - in principle - easy to add this support to an IDE like Visual Studio Code for example. It would be straightforward to set that up so that it always renders, displays any source file in some preferred language, even though it might be written in some other. Likewise it would be straightforward to have the system always save files in some preferred language too.

A user, teacher or student who wanted to work in say say Danish or just discuss some existing code in a class, using Danish has that option, they could easily look at a codebase coded in English and see only Danish keywords, if they wanted to.

It is not for me to tell people how to work, what language they must use for writing code when a native option is available, it is not for me to "police" how a person or a team works, a programming language should not attempt to impose control over how people think and work.

All programming languages and environments can be used sensibly and intelligently or they can be abused, but that's not my call, I don't decide what works for someone else, my fears and insecurities are not theirs, if something isn't suitable for some work or project then don't use it.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 11, 2023, 03:12:30 pm
Allowing different language keywords will break a havoc in developing and maintaining tools which work at the source code level. Just do not do it.

Why? first only a single lexicon's keywords can be used in any file, it is not permitted to ever mix lexicons in a source file. The lexicon is specified in a compiler directive or can be overridden as a compiler option. Every source file uses one and only one lexicon and the default is English, if no directive or compile option is specified.

One must "opt in" to use other lexicons.

Any source file can be converted from any lexicon to any other using a simple utility (and class library by tool builders).

Owners of repositories can establish a check that all pull requests contain source in some defined lexicon, this is trivial to do with GitHub actions, every contributor would have to save their code with that lexicon.

A developer could work locally by seeing Bulgarian keywords but always have their files saved as English.

The analogy with fonts and colors in an IDE is good, nobody cares about how the source appears to a developer, all we care about is what is shared.

It is your language and project, so you can do whatever you want to. I have just expressed my opinion and concern here.

That's fine with me, this is a very unusual feature, I expect some push back for that reason.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 11, 2023, 09:20:01 pm
Incidentally my thread asking about uses for analog scopes we not the idiocy some here seemed to insinuate, to them I say try this thread (https://www.eevblog.com/forum/projects/t20347/).
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 12, 2023, 01:39:03 pm
edit:
waste of time, don't feed the troll
Title: Re: A new, hardware "oriented" programming language
Post by: Nominal Animal on January 12, 2023, 03:35:18 pm
To continue from my last post:

If you create a mechanism for users to define the mapping between token sequences and patterns, and compiler backend abstract syntax tree (https://en.wikipedia.org/wiki/Abstract_syntax_tree), you don't have to define the programming language at all: you simply let each user define their own.

A different mapping could be used from token sequences and patterns to bytecode runtimes, like Java and CIL (https://en.wikipedia.org/wiki/Common_Intermediate_Language).

In theory, this would allow the same code to be compiled for CIL and Java runtimes, or use GCC or LLVM backends to generate machine code, just by defining the mapping between structures (token sequences and patterns) to each target.

Additionally, similar to GitHub, you could have a centralized repository of projects, but continuing with the social responsibility, it could automatically scan the token mappings (to human-readable languages), and assign each token set a political correctness score and warn sensitive developers if negative words like "master", "slave", "daemon", etc. are used.  As increasing number of young people are sensitive to such issues, but still interested in coding, this would lower the inherent stigmatizing barriers in current software development communities.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 12, 2023, 03:39:18 pm
I'm looking at the way to best represent exception handling in this new language.

Like in C# and Java, accessing variables defined within the scope of the faulting block should likely be prohibited since the fault might have been associated with the instantiation of those variables.

Making 'catch' a keyword statement mean's we could write this as:

Code: [Select]
procedure English (X)

declare counter binary(15);
 
        counter = FACE B034 D85F BAD7:H;

if counter > 0 then
   call retour;
else
   return;
end;

loop while (a > b)
   go to place;
end;

catch;
          /* cannot reference container's scope, i.e. cannot reference 'counter'
end;

begin;

   dcl array(x) binary(15);

   // do stuff

   catch;
              // cannot reference 'array' but can reference 'counter'
              // since the exception arose in this 'begin' block.
   end;

end;

The idea here is that a 'catch' block is an optional statement inside another block, rather than a clause within a try-catch syntactic construct.

Any code can throw an exception, if there's a 'catch' in scope invoke it, if not, search up the stack looking for one.

In C#, Java and I presume C++, a 'catch' block must be attached to a 'try' block, whereas here we avoid the need for an explicit 'try', we let a 'procedure' block a 'begin' block and so on, have an optional, enclosed 'catch' block instead...


Title: Re: The Inpercalium Programming language - IPL
Post by: Nominal Animal on January 12, 2023, 06:45:29 pm
One facet that I've been thinking about is code comments, and how to properly facilitate those in a multicultural, multilingual project.

One option is to ban direct source comments –– the code speaks for itself, after all ––, and instead use hyperlinks to specific team-wide emails (using the message-id) or IM discussions (using the exact date and time).  When those involve people outside the immediate team, they will use either commonly-agreed language, the shared language between the teams, or the messages and emails will be translated to the languages involved.  Good development environments already support normal hyperlinks within source code, so making such hyperlinks is trivial.

There may be a temptation to sidestep such rules by using otherwise unused literal strings in the sources –– like in Python, where the first literal string in a class, method, or function, is used for documenting it (pydoc) ––, so some teams might wish to ban those as well, and instead use keys referring to a database entry (with each string identified by the key and the language).   At build time, the compiler can trivially check which keys are used, and list those; which also makes localization and language version completeness easy to track.
Title: Re: The Inpercalium Programming language - IPL
Post by: tggzzz on January 12, 2023, 07:08:11 pm
–– the code speaks for itself, after all ––

No, it doesn't, except for trivial stuff.

In particular these things cannot be visible in the source code:

If all that is in a separate document, then you have even more chance of, ahem, "asynchronism" than with comments in the source code :)

Please don't mutter something about TDD. Usually the tests are the bare minimum crap, are unlikely to be comprehensive, and cannot prove that the code work. But you, at least, know that :)
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 12, 2023, 07:51:50 pm
Agree with tggzzz here.

You may write external documents to explain, for instance, a very complex algorithm. But otherwise, comments in the code still stand a much better chance of being in sync and being relevant. Besides, engineers usually hate writing documents, so adding some relevant comments in the code is a better option, they usually do not hate doing this as much. Of course the comments must be relevant, so as said above, comment the why rather than the how, and the how only sparingly and only if it's really non-trivial.

Write documents for specifications though. Oh, specs, you know, the bad word. Who needs specs, right? Just write some code and see if it passes tests. :-DD
Yes, TDD is (usually) bullshit and a misnomer. Testing is good. But specs should drive development, not tests. And what tells you that you're gonna do a better job at implementing a test for some spec, than at implementing the functionality itself?
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 12, 2023, 08:51:04 pm
Agree with tggzzz here.

You may write external documents to explain, for instance, a very complex algorithm. But otherwise, comments in the code still stand a much better chance of being in sync and being relevant. Besides, engineers usually hate writing documents, so adding some relevant comments in the code is a better option, they usually do not hate doing this as much. Of course the comments must be relevant, so as said above, comment the why rather than the how, and the how only sparingly and only if it's really non-trivial.

Write documents for specifications though. Oh, specs, you know, the bad word. Who needs specs, right? Just write some code and see if it passes tests. :-DD
Yes, TDD is (usually) bullshit and a misnomer. Testing is good. But specs should drive development, not tests. And what tells you that you're gonna do a better job at implementing a test for some spec, than at implementing the functionality itself?

I worked at a company once which did not permit comments. I did not share that view, but it was what it was, these were no lightweights either, amongst the sharpest developers I ever worked with.

I couldn't agree more about specifications, having been a programmer for decades and also a formal (hand off the code) project manager I've seen how easily time (itself a resource) can get wasted, huge amounts of time.

The most basic reason for decent specs is that there are usually a dev team and a qa team, the dev team need to write code and the qa team need to write test cases, unless these two teams are using the same identical problem statement you're going to get pseudo defects and often lots of them and that costs time.


Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 12, 2023, 09:37:08 pm
Yes, TDD is (usually) bullshit and a misnomer. Testing is good. But specs should drive development, not tests. And what tells you that you're gonna do a better job at implementing a test for some spec, than at implementing the functionality itself?

The idea behind TDD isn't too bad: write a test that illustrates some part of the specification. It will fail, so write code until it doesn't fail. If done with good taste and intelligence, that can work - especially for simple additions to functionality. But when the ignorant religious zealots get hold of the concept, they make it useless: "the green light is on therefore it is working".

And it is "difficult" to write a test for the appallingly named "non-functional specifications" such as "downtime <5minutes per year" or "updates to X must be ACID".
Title: Re: The Imperium programming language - IPL
Post by: MIS42N on January 12, 2023, 11:33:42 pm
Agree with tggzzz here.

You may write external documents to explain, for instance, a very complex algorithm. But otherwise, comments in the code still stand a much better chance of being in sync and being relevant. Besides, engineers usually hate writing documents, so adding some relevant comments in the code is a better option, they usually do not hate doing this as much. Of course the comments must be relevant, so as said above, comment the why rather than the how, and the how only sparingly and only if it's really non-trivial.

Absolutely.

 '–– the code speaks for itself, after all ––' yes, it does say what it does but nothing about why. I cannot conceive why "I worked at a company once which did not permit comments". Assembler is more obtuse than most languages, I find documentation almost essential. Take the following subroutine - in any language its purpose might be divined from its name, what it is doing (multiply, divide, convert to BCD) is very clear, but why it does what it does would be very difficult to work out. In my opinion documentation is very important and any language should make it easy to do it in whatever way is convenient - blocks, one liner, etc.:
Code: [Select]
; -- control volts to BCD --
; Converts the 24 bit control volt value to microvolt
; It is assumed that PWM 0x000000 = 0 Volt PWM 0xFA0000 = 5 Volt
; 0xFA0000 = 16,384,000.
; The fraction 112/367 is within 8uV at full scale, the integers
; used are limited by the 32 bit signed arithmetic routines.
; The value is not intended to be accurate, the source of the
; control voltage is unlikely to be perfect.
; Input is already pushed onto the stack
; Output is delivered to PackedD
CV2BCD:
    CALL    PushLit
    data    D'112'
    CALL    multiply
    CALL    PushLit
    data    D'367'
    CALL    divide
TI2BCD: ; entry used to convert stack to BCD
    CALL    Pop4
    data    REGA
    CALL    BIN2BCD
    RETURN
Title: Re: The Intercal programming language - IPL
Post by: Nominal Animal on January 13, 2023, 12:02:35 am
I disagree.  After all, OP's purpose is to create a programming language that is concise but descriptive enough to not need comments, because users can choose the language (and perhaps even the keywords they want to use).

I can envision this being a transition path from PL/I, IPL, COBOL, FORTRAN 77, various domain-specific languages based on SKI combinator calculus, BASIC, SAKO, Qalb, Fjölnir, Malbolge, and others, into proper modern CIL-based programming languages, with minimal disruption or re-education needed for the developers.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 13, 2023, 12:16:17 am
Agree with tggzzz here.

You may write external documents to explain, for instance, a very complex algorithm. But otherwise, comments in the code still stand a much better chance of being in sync and being relevant. Besides, engineers usually hate writing documents, so adding some relevant comments in the code is a better option, they usually do not hate doing this as much. Of course the comments must be relevant, so as said above, comment the why rather than the how, and the how only sparingly and only if it's really non-trivial.

Absolutely.

 '–– the code speaks for itself, after all ––' yes, it does say what it does but nothing about why. I cannot conceive why "I worked at a company once which did not permit comments". Assembler is more obtuse than most languages, I find documentation almost essential. Take the following subroutine - in any language its purpose might be divined from its name, what it is doing (multiply, divide, convert to BCD) is very clear, but why it does what it does would be very difficult to work out. In my opinion documentation is very important and any language should make it easy to do it in whatever way is convenient - blocks, one liner, etc.:
Code: [Select]
; -- control volts to BCD --
; Converts the 24 bit control volt value to microvolt
; It is assumed that PWM 0x000000 = 0 Volt PWM 0xFA0000 = 5 Volt
; 0xFA0000 = 16,384,000.
; The fraction 112/367 is within 8uV at full scale, the integers
; used are limited by the 32 bit signed arithmetic routines.
; The value is not intended to be accurate, the source of the
; control voltage is unlikely to be perfect.
; Input is already pushed onto the stack
; Output is delivered to PackedD
CV2BCD:
    CALL    PushLit
    data    D'112'
    CALL    multiply
    CALL    PushLit
    data    D'367'
    CALL    divide
TI2BCD: ; entry used to convert stack to BCD
    CALL    Pop4
    data    REGA
    CALL    BIN2BCD
    RETURN

Well said, however in my formative years, learning "programming" on IBM mainframes, I was beaten if I did not comment. Later I developed a reputation for plentiful, constructive comments.

I still, often, begin to write non trivial code by writing the comments first, little blocks of comments that I then "back fill" with code.

But at Faithlife (https://www.logos.com/grow/what-is-logos-bible-software/) many years later, they "do not do" comments. Their reasons were things like "the comments never get updated when the code is changed" and "you can compile the code but not the comments" which is - like it or not - very true.

I did not agree with their overall approach but it was entrenched in them and they did raise some very relevant points, they are first rate developers but I disagreed about this (and some other things).

A comment that no longer matches the code, is worse that no comment at all - this can be true, so it's far from clear what to do really.



Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 13, 2023, 02:42:30 am
Organizations more or less all have their own set of rules, and whether they are good or lame, you just have to follow the rules. Discussing those rules is often pointless: if you work there, you don't have a choice, and if you don't work there, you may not have a full view of the rationale behind the rules.

The risk of comments not being up to date is one of the most common arguments I've heard for defending the zero-comment approach.

You may wonder if a developer that fails to keep comments up to date can be trusted on their code, though.

And then, as we said above, if you comment the 'why' rather than the 'how', chances of having comments not up to date are drastically lower. While changing the 'how' can happen pretty frequently (and thus, the probability of having obsolete comments is high if they are about the 'how'), the chances of changing the 'why' is much more uncommon, and if you change the 'why', then usually that means a rewrite big enough that you'd almost have to do it on purpose to keep old, obsolete comments about the 'why'. Just my 2 cents.

Finally, that's also what code reviews are for.
Title: Re: The Intercal programming language - IPL
Post by: tggzzz on January 13, 2023, 09:33:33 am
I disagree.  After all, OP's purpose is to create a programming language that is concise but descriptive enough to not need comments, because users can choose the language (and perhaps even the keywords they want to use).

If a programming language can express why rather than what, then that will be a truly remarkable advance, and I will be very interested in the language. I suspect those concepts would find their way into many programming languages in the future.

But I'm not going to hold my breath, either for this language nor future languages :)
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 13, 2023, 03:15:33 pm
Organizations more or less all have their own set of rules, and whether they are good or lame, you just have to follow the rules. Discussing those rules is often pointless: if you work there, you don't have a choice, and if you don't work there, you may not have a full view of the rationale behind the rules.

The risk of comments not being up to date is one of the most common arguments I've heard for defending the zero-comment approach.

You may wonder if a developer that fails to keep comments up to date can be trusted on their code, though.

And then, as we said above, if you comment the 'why' rather than the 'how', chances of having comments not up to date are drastically lower. While changing the 'how' can happen pretty frequently (and thus, the probability of having obsolete comments is high if they are about the 'how'), the chances of changing the 'why' is much more uncommon, and if you change the 'why', then usually that means a rewrite big enough that you'd almost have to do it on purpose to keep old, obsolete comments about the 'why'. Just my 2 cents.

Finally, that's also what code reviews are for.

Well said, this is why I personally hold the view that designing, writing, maintaining and enhancing software is best described as a "craft" rather than "science" or "engineering", I liken it to architecture myself, there are aspects that are based in physics and mechanical engineering yet there are also aspects that are aesthetics, style, atmosphere, art and so on - both aspects are essential.
Title: Re: The New Intercal Programming Language - IPL
Post by: Nominal Animal on January 13, 2023, 06:37:22 pm
I personally hold the view that designing, writing, maintaining and enhancing software is best described as a "craft" rather than "science" or "engineering", I liken it to architecture myself, there are aspects that are based in physics and mechanical engineering yet there are also aspects that are aesthetics, style, atmosphere, art and so on - both aspects are essential.
Spoken like a true Windows-only C# programmer who has zero idea about verifiability, robustness, or security.  Congratulations!
I don't recall when I last read such idiotic drivel, and I often read people spouting conspiracy theories at Rense and AboveTopSecret.

It's like listening to a bridge-builder describe how they scoff at stuff like engineering calculations, and instead prefers to call themselves a craftsman.  Would you trust such a bridge?  I wouldn't.
The world is already chock full of crappy software written by craftsmen and monkeys, ignoring all engineering principles and scientific thinking and research.
We do not need a single more.  Please!

My last few posts have been an experiment.  As I've mentioned in other discussions, I sometimes entertain ideas and concepts I do not believe in at all, like conspiracy theories, just to find out where they lead to, and how their proponents see the world around them.  In this case, I inverted everything I have learned in practice about teamwork, portability, maintainability, and so on, and wanted to see if OP has any pushback to the known non-working ideas I would describe.

The concepts I described:None of these work in practice, but are enticing if and only if one does not consider the implementation or functionality and the related consequences.



Human-readable text is converted to tokens using lexical analysis (https://en.wikipedia.org/wiki/Lexical_analysis).  The sequence of tokens are converted to some form of abstract syntax, usually an abstract syntax tree (https://en.wikipedia.org/wiki/Abstract_syntax_tree) (AST), by a parser (https://en.wikipedia.org/wiki/Parsing) performing syntactic analysis.  Many – if not most – interpreters and compiler input frontends use C code generated from a Backus-Naur form specification by flex (https://en.wikipedia.org/wiki/Flex_(lexical_analyser_generator)) and Yacc (https://en.wikipedia.org/wiki/Yacc) or GNU Bison (https://en.wikipedia.org/wiki/GNU_Bison).  The latter can also generate lexers-parsers for C++ and even Java.

It is the mapping from this abstract form, to the abstract forms that can be generated into machine code, that defines a programming language.  For example, there is no such concept as a "function" in machine code.  There are subroutines, and an instruction that will call a subroutine, which ends at some kind of a return statement, which causes the execution to continue at the instruction following the call.  So, to map a function definition to a form comprehensible to the backend, you need to include things like "include this symbol (the name) in local symbol table using type 'function', ..." and so on.
There is a lot of scientific research on how this can be done.  Leaving it to users is like giving an English-only speaking student a Hungarian dictionary, and telling them to write a novel in Hungarian.  Perhaps one can consider writing to be a craft, but unless you know the rules and science, all you generate is word soup –– or horribly crappy software.

The above means that syntax and grammar, including the selection of reserved keywords and operators –– all tokens recognized ––, is something that can and should be finalized last, just before the language specification is released.  Until then, the syntax and grammar can be varied at will, without affecting the core of the language: exactly what it does, i.e. the mapping between the language-specific AST and the language-and-architecture independent AST used by the code generating backend.  Starting at syntax and grammar is like deciding to create a new spoken language like Esperanto, and prioritizing deciding whether the language will use Oxford comma (https://en.wikipedia.org/wiki/Serial_comma) or not.  Utterly ridiculous and irrelevant!  Such details are best resolved last, not first.

A lot of the practical progress in the last decade or so in computer science has been in optimization at both the frontend (language-specific AST) and backend (architecture and language-independent AST), using (old) discoveries in graph theory, and basically the exponentially increasing computational power allowing slower/more computationally costly algorithms to be used to simplify the ASTs.  Using resources like Compiler Explorer (https://godbolt.org/) (godbolt.org) to examine the machine code generated by different compilers and different optimization levels, shows both how much work has already been done here, but also how much work there is still to be done with the current compilers.

For the GNU Compiler Collection, the backend language is called GENERIC (and GIMPLE is its simplified version, it being possible to map GENERIC to GIMPLE, splitting certain constructs into multiple ones); see here (https://gcc.gnu.org/onlinedocs/gccint/GENERIC.html).  LLVM is the backend language, although 'intermediate representation' (IR) is used when 'LLVM' would be ambiguous; see here (https://llvm.org/docs/LangRef.html).

In a typical complex application/service/firmware build, there are multiple stages.  It is not only the 'compiler' that accesses the source code, various tools from code generators (like flex and yacc/bison) to static code analysis (https://en.wikipedia.org/wiki/Static_program_analysis) to dependency checkers (to speed up builds) and so on.  Having the exact source code in plain text formats allows the widest possible set of tools here; including project-specific custom tools (like the many ones used in building the Linux kernel, for example; from configurators to formatting checks to locking scheme verification).

Even script languages like Python use the plain text source, and 'compile' to internal token/bytecode format dynamically, although it can also store the binary representation of the source code in .pyc files if so requested, to reduce load latencies; even then, it does verify that the binary file is more recent than the text source file.  This is not just an arbitrary decision, but a decision based on lots and lots of experience.  Plain text sources will always be more versatile (and therefore useful to users and developers) than binary formats.

While ZIP archives sound good, they are basically write-only.  To modify just one file, basically the entire archive has to be recompressed; or at least recopied.  Thus, it makes zero sense to use any kind of packed archive format for source code during development, because it simply slows down I/O.  Using a sensible directory/folder structure will always work better.  Learning how distributed source management tools like Git (https://git-scm.com/) work, will give you a much better picture of the existing knowledge and experience in this subfield.

Gödel number (https://en.wikipedia.org/wiki/G%C3%B6del_numbering) was an indication that I'm being surreal; I couldn't just be serious, because that could actually lead people who find my contributions worthwhile reading, astray.  It means that instead of a sequence of natural numbers (identifying the tokens), the source code is encoded as a single huge natural number: a product of the prime powers of the token.  The i'th multiplier is the i'th prime raised to the power of the natural number representing the i'th token.  While encoding is reasonably fast, decoding involves prime factorization of a natural number possibly hundreds of thousands of bits long.  There is absolutely no reason to do this!

Vendor lock-in and vendor control of our projects is something you might not be afraid of at all as a developer, until it bites you.  Then you'll avoid it like the plague it is.  The vendor owning encryption keys (and essentially having access to all clients' encrypted sources) is probably the scariest scenario of rent-seeking vendor behaviour that I can think of.  No sane experienced developer will subject themselves to it, it is simply just too risky: years of work may be irretrievably lost because of 'minor vendor policy change'.

As described by others, source code comments are worthless if they describe what or how the code does what it does.  We know from decades-old projects that are still alive and maintainable, that the most useful and practical comments are those that describe developer intent and assumptions.  For example, if you refactor a function, you typically should not need to change the comment describing the purpose and assumptions of the function, unless the assumptions (for example locking scheme) changes.

I have done my best work in teams with wildly differing members.  I have also taught basic computer skills (to both youngsters at the turn of the century, as well as to teachers older than myself), mentored dozens of programmers (from just about every continent except Antarctica), worked for two different universities with lots of exchange students (so much so that typically even lunch discussions were in English, to not exclude anyone present – it was especially funny when we noticed everybody present was fluent in Finnish).  I have lead teams, interviewed and hired people.
In my experience, synergistic collaboration – where teams become more than just the sum of their parts – requires effective, clear, honest, direct, and free discussion, debate, and even argument.  Everybody participating must agree that arguments/debates are done at the logical level, and must only concern things; and peoples' personalities must be kept out of it.  The rules must be clear, easily checked, and apply to everyone equally.
Intersectionalist and cultural relativist approach where each person is governed by rules depending on non-technical aspects of their personality will not work, and will destroy teams and significantly degrade their work product and performance, because humans are social animals with inherent, biological concept of fairness, specifically procedural justice (https://en.wikipedia.org/wiki/Procedural_justice) and distributive justice (https://en.wikipedia.org/wiki/Distributive_justice) (as observed in several animal species also).
Competitiveness based on meritocracy (https://en.wikipedia.org/wiki/Meritocracy) is useful, but may lead to internal conflict within a team, so I recommend larger emphasis on team rewards than individual rewards in most cases; it does vary on the situation, though.

In simple terms, the current fad of political correctness is dysfunctional in development environments.  It leads to reduced discussions (because all discussions have a risk of offense, so avoiding discussions is always the safer option in PC-conscious environments).  We can see this in how certain open source projects (funded by Red Hat/IBM in particular) are already asking for isolation from users, especially in regards to problem reports.  They would prefer to receive only automatically generated problem reports, even though they contain far less information.

The period of stagnation in the GCC community in the first decade of the current century was largely because the key GCC developers refused to consider anything external to the project, unless the reporter had at least a PhD in Computer Science.  I do not know how those developers were retired, whether it was mutually agreed upon, or some kind of machination between the GNU foundation and the GCC project, and how much of the change was driven by the eventual shift from C to C++, but overall, it is a perfect example of how isolated open source developers can kill projects.

Also, the longest-lived, most 'successful' (for various definitions of 'successful') projects, have traditionally been led by 'dictatorial' leaders, often with analogs of 'advisory councils'.  As usual, the leader mostly sets the social atmosphere and long-term goals in the project, but also seems to be the quality enforcer: responsible for rejecting sub-standard suggestions.  For the Linux kernel, Linus Torvalds has been berated for his harsh language, not for his 'autocratic control'.  Democratic processes are easily subverted (as seen by for example the Debian project regarding init systems; just read the discussions prior to the general resolution, and compare to the consequences observed today), and many respectable developers occasionally produce utter garbage that deserves to be rejected.  See for example Greg KH's suggested kdbus subsystem, discussed at the linux-kernel mailing list.  (RedHat proponents at the time believed that if they got Greg KH to suggest it, his authority would be enough to silence any objections.  No, that didn't work, because the contribution was crap.  The good thing about it was that Greg KH just accepted the rejection, and that was it: it did not, and should not, affect his later contributions, because contributions are and must be evaluated on their own merits, and not based on who submitted them.  He is still very highly regarded Linux kernel developer, and I personally 'trust' him; nevertheless, whatever he submits as a new feature, requires exactly the same scrutiny as any other submission.  This is how a divergent, distributed team functions well.)

I could also describe why I find complex runtimes like Java and CIL objectionable, but I'm not sure it is worth going into here.  Simply put, they are an unnecessary abstraction layer that attempts to gloss over differences in environments and operating systems and processor architectures.  In my experience, this leads to the phenomenon historically referred to as "writing FORTRAN code in any programming language".  While it makes development available to less skilled developers, and therefore cheaper, they really bring no real benefits (compared to native code written in a comparable language) to the end users.
I just care more about end users than developers myself; specifically, the end users that contribute their time, effort, or money to my open source projects; and the paying end users for the proprietary projects I've contributed to.  With sufficient skills and ability to implement anything I want in about a dozen programming languages, the Java RE and MS CIL are unnecessary and unwanted abstraction layers to me: waste of resources.

To repeat once again: to develop a new language, you need to consider both how the concepts and algorithms are implemented in that language, and what kind of machine code one should expect them to generate.  Starting from the syntax and grammar is starting from how the code should look like; something that logically should be decided last, just before the language specification itself is finalized.  You need to ask yourself "Why would I want to implement algorithm X or system Y in this language, as opposed to some other language?".  If your answer is "Because it's better", you do not have the skill in design, logic, and software engineering to pull it off, sorry; and are just daydreaming about appearances without any actual content.
Title: Re: The New Intercal Programming Language - IPL
Post by: tggzzz on January 13, 2023, 11:01:22 pm
I personally hold the view that designing, writing, maintaining and enhancing software is best described as a "craft" rather than "science" or "engineering", I liken it to architecture myself, there are aspects that are based in physics and mechanical engineering yet there are also aspects that are aesthetics, style, atmosphere, art and so on - both aspects are essential.
Spoken like a true Windows-only C# programmer who has zero idea about verifiability, robustness, or security.  Congratulations!
I don't recall when I last read such idiotic drivel, and I often read people spouting conspiracy theories at Rense and AboveTopSecret.

It's like listening to a bridge-builder describe how they scoff at stuff like engineering calculations, and instead prefers to call themselves a craftsman.  Would you trust such a bridge?  I wouldn't.
The world is already chock full of crappy software written by craftsmen and monkeys, ignoring all engineering principles and scientific thinking and research.
We do not need a single more.  Please!

I had very similar thoughts, the major difference being that C# is no worse than other languages.
Title: Re: The New Intercal Programming Language - IPL
Post by: tggzzz on January 13, 2023, 11:13:38 pm
Snipped many sensible poins...

To repeat once again: to develop a new language, you need to consider both how the concepts and algorithms are implemented in that language, and what kind of machine code one should expect them to generate.  Starting from the syntax and grammar is starting from how the code should look like; something that logically should be decided last, just before the language specification itself is finalized.  You need to ask yourself "Why would I want to implement algorithm X or system Y in this language, as opposed to some other language?".  If your answer is "Because it's better", you do not have the skill in design, logic, and software engineering to pull it off, sorry; and are just daydreaming about appearances without any actual content.

Precisely.

The OP appears not to want to use those concepts as a starting point, bit to concentrate on what interests them. That's reasonable, iff the limitations are understood and accepted.
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 14, 2023, 12:02:34 am
@Nominal Animal
Java ... my Java-cup says "program once, use many"...and I laugh every time I use it to drink my coffee.

It was the Java v1.0 motd when I bought that cup in 2001, and I found that it is somehow a good cup that keeps the temperature of the coffee (so the box wasn't lying when I bought it), just ... I keep it on my desk, but I no longer trust what it says about programming  :o
Title: Re: The New Intercal Programming Language - IPL
Post by: Nominal Animal on January 14, 2023, 12:08:16 am
I had very similar thoughts, the major difference being that C# is no worse than other languages.
It is similar to Java in that it compiles to CIL (https://en.wikipedia.org/wiki/Common_Intermediate_Language), a stack-based bytecode interpreter, which is touted as "portable".  We all know how portable Java is, and how portable .Net/Mono is: not very, when you start trying to get actual work done, instead of simple examples.  Instead of acknowledging differences between operating systems and hardware architectures, it applies a heavy (again, bytecode interpreter/JIT compiler) abstraction layer to hide them.  For this reason alone, C# is not a language one should base their understanding of programming languages on.

(We can nitpick about NGEN (https://en.wikipedia.org/wiki/Native_Image_Generator), though.)

Portability and adaptation to different architectures, and understanding the different approaches (paradigms) different languages have, is utterly paramount for understanding software engineering in a way that one can apply in more than one particular niche (Windows C# programming).  C# is designed to avoid having to do that.

If we ignore the single-vendor control, the reliance on CIL, and just look at the language specification, say the latest version (C# 6.0) standardized by ECMA in ECMA-334.pdf (https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-334.pdf) –– noting that Microsoft published it in 2015, ECMA-334 was published in 2022, and Microsoft has already published C# 11.0 –– you could compare it to older versions of C++.  In particular, it does not have specific-size integers, and instead codifies that 'short' is 16-bit, 'int' is 32-bit, 'long' 64-bit, and the 'char' type uses UTF-16; i.e. IP32 or LP64.
In short, meh.

Java ... my Java-cup says "program once, use many"...and I laugh every time I use it to drink my coffee.
Exactly.

If we do not learn from the past, we're doomed to repeat the same mistakes, as well.
Title: Re: The New Intercal Programming Language - IPL
Post by: MIS42N on January 14, 2023, 01:55:01 am
My last few posts have been an experiment.  As I've mentioned in other discussions, I sometimes entertain ideas and concepts I do not believe in at all, like conspiracy theories, just to find out where they lead to, and how their proponents see the world around them.  In this case, I inverted everything I have learned in practice about teamwork, portability, maintainability, and so on, and wanted to see if OP has any pushback to the known non-working ideas I would describe.
Aha. I looked at some of those and thought WTF? I want to be constructive so try not to malign other posts.

In a way, programming is a craft. We walk the fine line between constraints, some being size of code, efficiency of code, cost of production, cost of maintenance, tolerance of bugs (e.g a game v. medical equipment). When some operation needs doing 20 times, if code space is not an issue but speed is then make it a macro and put it in line 20 times. If speed is not an issue, make a loop. Programming languages rarely make that choice for you. Use a binary search or a linear search of an ordered array? use hash tables or original data? They are not necessarily yes/no answers, a good programmer will make a good choice - the result of experience - is that not the hallmark of a craft?

But yes, it is less of a craft today when processing power and memory are cheap and plentiful. It is easy to use something that works rather than something that works better. Especially if the something better is not familiar to others. I had the joy of reducing the run time of an application from 3 hours to just over 1 minute (and won a bet for a beer for the effort). Today it may be the difference between 20 seconds and a tenth of a second, who cares?
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 14, 2023, 02:08:54 am
https://www.youtube.com/watch?v=AbgsfeGvg3E (https://www.youtube.com/watch?v=AbgsfeGvg3E)
Title: Re: The New Intercal Programming Language - IPL
Post by: brucehoult on January 14, 2023, 02:50:51 am
I had very similar thoughts, the major difference being that C# is no worse than other languages.
It is similar to Java in that it compiles to CIL (https://en.wikipedia.org/wiki/Common_Intermediate_Language), a stack-based bytecode interpreter, which is touted as "portable".  We all know how portable Java is, and how portable .Net/Mono is: not very, when you start trying to get actual work done, instead of simple examples.  Instead of acknowledging differences between operating systems and hardware architectures, it applies a heavy (again, bytecode interpreter/JIT compiler) abstraction layer to hide them.  For this reason alone, C# is not a language one should base their understanding of programming languages on.

Wanting to define the result of every construct and getting bit-identical results from every program, no matter what machine or environment it is run on, are not evil goals. They are simply goals where getting maximum efficiency, performance, and execution predictability on any given machine is not a priority.

Modern machines are so fast that this is often fine.

Lots of people use things much more inefficient than Java or C#. For example Python, which gives not just 80% or 50% of the performance of native code, but 3% to 5%.

This is all completely at odds with the concept of cycle-accurate code on a PIC, of course.
Title: Re: The New Intercal Programming Language - IPL
Post by: Nominal Animal on January 14, 2023, 04:28:27 am
I had very similar thoughts, the major difference being that C# is no worse than other languages.
It is similar to Java in that it compiles to CIL (https://en.wikipedia.org/wiki/Common_Intermediate_Language), a stack-based bytecode interpreter, which is touted as "portable".  We all know how portable Java is, and how portable .Net/Mono is: not very, when you start trying to get actual work done, instead of simple examples.  Instead of acknowledging differences between operating systems and hardware architectures, it applies a heavy (again, bytecode interpreter/JIT compiler) abstraction layer to hide them.  For this reason alone, C# is not a language one should base their understanding of programming languages on.

Wanting to define the result of every construct and getting bit-identical results from every program, no matter what machine or environment it is run on, are not evil goals.
Did I even imply they were?  I am only describing the reasons why I believe C# is not sufficient or suitable as the basis for ones understanding for designing a programming language.

Lots of people use things much more inefficient than Java or C#.
Yes, I use such things as well, but how is that supposed to be relevant here?

Do you seriously believe that people who only have used Java or C# have a well-rounded understanding of how programming languages and environments work?  Because that is what your reply seems to imply.

My own experience indicates they simply propagate the inefficiencies and domain-specific patterns as if they were universal (which they are not), and simply call the result better than what has been done before, without any scientific or engineering basis on such qualifications.
Title: Re: The New Intercal Programming Language - IPL
Post by: tggzzz on January 14, 2023, 09:38:40 am
I had very similar thoughts, the major difference being that C# is no worse than other languages.
It is similar to Java in that it compiles to CIL (https://en.wikipedia.org/wiki/Common_Intermediate_Language), a stack-based bytecode interpreter, which is touted as "portable".  We all know how portable Java is, and how portable .Net/Mono is: not very, when you start trying to get actual work done, instead of simple examples.  Instead of acknowledging differences between operating systems and hardware architectures, it applies a heavy (again, bytecode interpreter/JIT compiler) abstraction layer to hide them. 

Shortly before C# was released, Anders Hejlsberg gave a talk about it at work. We were all underwhelmed, regarded it at a typical me-too-entend-embrace-extinguish MS ploy. I never programmed in it, and never regretted it. I have never had much faith in Mono either :)

Quote
For this reason alone, C# is not a language one should base their understanding of programming languages on.

At the language level, C# is Java with added insecurities.

At the implementation level, C# is Java with longer application installation times due to static optimisation, and the inability to optimise based on how the code is interacting with the data.

Beyond that, it is better to ignore the details, and free up your brain to concentrate on significantly different languages.

Quote
Portability and adaptation to different architectures, and understanding the different approaches (paradigms) different languages have, is utterly paramount for understanding software engineering in a way that one can apply in more than one particular niche (Windows C# programming).  C# is designed to avoid having to do that.

All modern languages are designed to conceal the underlying architecture.

Quote
If we ignore the single-vendor control, the reliance on CIL, and just look at the language specification, say the latest version (C# 6.0) standardized by ECMA in ECMA-334.pdf (https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-334.pdf) –– noting that Microsoft published it in 2015, ECMA-334 was published in 2022, and Microsoft has already published C# 11.0 –– you could compare it to older versions of C++.  In particular, it does not have specific-size integers, and instead codifies that 'short' is 16-bit, 'int' is 32-bit, 'long' 64-bit, and the 'char' type uses UTF-16; i.e. IP32 or LP64.
In short, meh.

Such delays in standardisation are, ahem, standard. But there are worse phenomena, e.g. I remember a usenet post triumphantly announcing the first complete C++ compiler six years after the spec was released. That alone confirmed my earlier decision that C++ was fundamentally the wrong direction!

I didn't know about that integer issue, because I decided not to trouble my brain with C# details. I presume it was a consequence of the security holes in C# programs containing C/C++ components.

Quote
Java ... my Java-cup says "program once, use many"...and I laugh every time I use it to drink my coffee.
Exactly.

If we do not learn from the past, we're doomed to repeat the same mistakes, as well.

Java's portability is far better than C/C++ :) How much that matters in any given case is up for discussion.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 14, 2023, 03:32:09 pm
String Literals

How important is a rich support for string literals in an MCU world? C# recently introduced an impressive innovation, is this useful? how important is this kind of feature?

https://www.stevefenton.co.uk/blog/2022/02/raw-string-literals-in-c/ (https://www.stevefenton.co.uk/blog/2022/02/raw-string-literals-in-c/)
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 14, 2023, 03:49:09 pm
String Literals

How important is a rich support for string literals in an MCU world? C# recently introduced an impressive innovation, is this useful? how important is this kind of feature?

https://www.stevefenton.co.uk/blog/2022/02/raw-string-literals-in-c/ (https://www.stevefenton.co.uk/blog/2022/02/raw-string-literals-in-c/)

"Feature" is the right word. Without "benefit" nobody will care.

The end result is easily achievable right now. Hence it is syntactic sugar (https://en.wikipedia.org/wiki/Syntactic_sugar#Criticism), not a benefit.

But no doubt those that go into paroxyisms of delight that the Edge browser may have rounded corners (https://www.techradar.com/news/microsoft-edge-could-go-for-rounded-corners-in-a-big-way-but-that-might-not-be-popular) (or not) will become ecstatic.
Title: Re: The Imperium programming language - IPL
Post by: Siwastaja on January 14, 2023, 03:51:12 pm
In MCU world, not only string literals, but ability to include a binary file would be actually handy. Something like

uint8_t coeffs[] =
#binary_include "coeffs.bin"

so that one does not need to write conversion tools from binary to "{0x12, 0x34"} etc.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 14, 2023, 04:00:42 pm
In MCU world, not only string literals, but ability to include a binary file would be actually handy. Something like

uint8_t coeffs[] =
#binary_include "coeffs.bin"

so that one does not need to write conversion tools from binary to "{0x12, 0x34"} etc.

Now that's an interesting idea, exactly the kind of feature suggestions I'm looking for!
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 14, 2023, 04:05:31 pm
String Literals

How important is a rich support for string literals in an MCU world? C# recently introduced an impressive innovation, is this useful? how important is this kind of feature?

https://www.stevefenton.co.uk/blog/2022/02/raw-string-literals-in-c/ (https://www.stevefenton.co.uk/blog/2022/02/raw-string-literals-in-c/)

"Feature" is the right word. Without "benefit" nobody will care.

The end result is easily achievable right now. Hence it is syntactic sugar (https://en.wikipedia.org/wiki/Syntactic_sugar#Criticism), not a benefit.

But no doubt those that go into paroxyisms of delight that the Edge browser may have rounded corners (https://www.techradar.com/news/microsoft-edge-could-go-for-rounded-corners-in-a-big-way-but-that-might-not-be-popular) (or not) will become ecstatic.

Actually its not easily achievable at all, take a look at this more in depth article.

https://blog.ndepend.com/c-11-raw-string-literals-explained/ (https://blog.ndepend.com/c-11-raw-string-literals-explained/)

(https://blog.ndepend.com/wp-content/uploads/Raw-String-Delimiter-must-be-on-its-own-line.png)

The number of prefix $ and " characters is variable.

Two $$ for example means single { } are treated as literal text, but {{ }} are interpreted as a runtime expression.

If one wants to embed " inside a string its easy:

"""This is an example of "Heading Label" we can use."""

If we needed to - ever - embed two quotes "" we can do this:

""""This is an example of ""Heading Label"" we can use.""""
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 14, 2023, 04:26:11 pm
In MCU world, not only string literals, but ability to include a binary file would be actually handy. Something like

uint8_t coeffs[] =
#binary_include "coeffs.bin"

so that one does not need to write conversion tools from binary to "{0x12, 0x34"} etc.

So a compile time initialization of static data via an external data file would be useful? I've never heard of this idea before, its extremely interesting.

I did already introduce this better notation too for numeric literals:

Code: [Select]
       counter = FACE B034 D85F BAD7:H;

So including a binary file could be done by converting the file (during compilation, the compiler would do it) to that kind of string...doing that at compile time eliminates any need for the code to have to do any kind of file IO as well, less code, less libraries...

How to express that kind of initialization though. The above was intended initially as just another way to express a numeric constant, I've not thought about the initialization side of this yet...

One could allow an array to be initialized just using that notation:

Code: [Select]
dcl coeffs(1024) bin(8);

coeffs = AB CE 7A 88 96 4E 3D.......:H;


Would expect there to be 1024 sets of 2-digit-hex values and that's it.

Whereas:

Code: [Select]
dcl coeffs(1024) bin(16);

coeffs = ABCE 7A88 964E 3D42 0004.......:H;

Would expect them as bocks of 4-digit-hex values.

The meaning is clear and the logic is basically to populate elements 1 thru 1024 with each successive constant, each space separated chunk would be regarded as the data for a single element.

So if we allow arrays (of primitive types) to be initialized that way, doing a "binclude" would be pretty straightforward for the compiler to do.

Code: [Select]
dcl coeffs(1024) bin(16);

coeffs =

#binclude "adc_table.h"

;

So to speak...I guess this falls under the preprocessor heading, something I posted very briefly about a couple days ago.

Title: Re: The Imperium programming language - IPL
Post by: magic on January 14, 2023, 06:47:18 pm
I hope this thread is not time wasted and the final spec will be ready by early April :)

In MCU world, not only string literals, but ability to include a binary file would be actually handy. Something like

uint8_t coeffs[] =
#binary_include "coeffs.bin"

so that one does not need to write conversion tools from binary to "{0x12, 0x34"} etc.
Pretty sure that somewhere on stackoverflow there is a stupid little script which achieves this with binutils; then it should be a simple matter of linking the resulting .o into your project and
Code: [Select]
coeffs.h:

extern uint8_t *coeffs;
extern size_t coeffs_size;
Title: Re: The Imperium programming language - IPL
Post by: ataradov on January 14, 2023, 06:56:39 pm
I do that binary inclusion with GCC for some project. It works, but I can't say it is pretty. Here is what I do for that

In the command line pass the files you want to be included as raw byte arrays:
Code: [Select]
RESOURCES = \
  res/icon.png \
  res/image_1.png \
  res/image_2.png \

LIBS += -Wl,--format=binary -Wl,--start-group $(RESOURCES) -Wl,--end-group -Wl,--format=default

And then in the code declare the symbols corresponding to those file:
Code: [Select]
extern uint8_t _binary_res_image_1_png_start;
extern uint8_t _binary_res_image_1_png_end;

Then you can get a pointer and a size as this:
Code: [Select]
uint8_t *data = &_binary_res_image_1_png_start;
int     size  = (int)(&_binary_res_image_1_png_end - &_binary_res_image_1_png_start);

It is better than using external tools, for sure. And in practice I don't find myself needing to do this a lot, so not a huge issue.
Title: Re: The Imperium programming language - IPL
Post by: Siwastaja on January 14, 2023, 07:23:45 pm
LIBS += -Wl,--format=binary -Wl,--start-group $(RESOURCES) -Wl,--end-group -Wl,--format=default

Excellent tip, and well demonstrates how the ecosystem and tooling is as important as the language itself. And that is the strength of C, despite all the shortcomings of the language itself.
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 14, 2023, 07:26:55 pm
I do that binary inclusion with GCC for some project. It works, but I can't say it is pretty. Here is what I do for that

In the command line pass the files you want to be included as raw byte arrays:
Code: [Select]
RESOURCES = \
  res/icon.png \
  res/image_1.png \
  res/image_2.png \

LIBS += -Wl,--format=binary -Wl,--start-group $(RESOURCES) -Wl,--end-group -Wl,--format=default

And then in the code declare the symbols corresponding to those file:
Code: [Select]
extern uint8_t _binary_res_image_1_png_start;
extern uint8_t _binary_res_image_1_png_end;

Then you can get a pointer and a size as this:
Code: [Select]
uint8_t *data = &_binary_res_image_1_png_start;
int     size  = (int)(&_binary_res_image_1_png_end - &_binary_res_image_1_png_start);

It is better than using external tools, for sure. And in practice I don't find myself needing to do this a lot, so not a huge issue.

I am using this as well. It's the easiest way of including "resources" in your programs.

I've written a small library to make it easier to use, it does essentially what you are showing, but allows declaring and accessing 'resource' objects easily. Very lightweight.
Title: Re: The Imperium programming language - IPL
Post by: magic on January 14, 2023, 07:39:44 pm
 :-+

I knew it has to be a solved problem ;D
It even gives you those _size symbols, so you don't need to calculate blob sizes manually.

Code: [Select]
$ ld -o opa2604.o --format=binary --start-group opa2604_30v.png opa2604_48v.png --end-group
ld: warning: cannot find entry symbol _start; not setting start address
$ ls -l opa2604*
-rw-r--r-- 1 m users 45920 2019-03-27  opa2604_30v.png
-rw-r--r-- 1 m users 44319 2019-03-27  opa2604_48v.png
-rwxr-xr-x 1 m users 94956 01-14 20:33 opa2604.o
$ objdump -t opa2604.o

opa2604.o:     file format elf32-i386

SYMBOL TABLE:
08049000 l    d  .data  00000000 .data
00000000         *UND*  00000000 _start
0805f07f g       .data  00000000 __bss_start
08049000 g       .data  00000000 _binary_opa2604_30v_png_start
0805f07f g       .data  00000000 _binary_opa2604_48v_png_end
0805f07f g       .data  00000000 _edata
0805f080 g       .data  00000000 _end
08054360 g       .data  00000000 _binary_opa2604_30v_png_end
0000ad1f g       *ABS*  00000000 _binary_opa2604_48v_png_size
08054360 g       .data  00000000 _binary_opa2604_48v_png_start
0000b360 g       *ABS*  00000000 _binary_opa2604_30v_png_size


edit
It even gives you those _size symbols, so you don't need to calculate blob sizes manually.
But they are not ordinary variables and trying to use them as such causes segfaults.
You need to take the address of that symbol to get the size of its corresponding blob.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 15, 2023, 01:56:18 pm
So injecting binary static data in this manner is deemed sufficient? an ability have a compiler do this simply by compiling the code, wouldn't be that helpful?
Title: Re: The Imperium programming language - IPL
Post by: ataradov on January 15, 2023, 05:41:10 pm
In a newly designed language I would do something better, of course. But this is such a minor thing that it would not be a consideration for switching languages or anything like that.
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 15, 2023, 08:02:07 pm
So injecting binary static data in this manner is deemed sufficient? an ability have a compiler do this simply by compiling the code, wouldn't be that helpful?

It's not "injecting" data any more than a compiler injects data with compiled object code. It's the same thing.

I actually kinda like it. It's generating object files from binary files, adding this in the programming language itself doesn't add anything much, except possibly making it cuter, but this can be made cuter by wraoping it with a few lines of code (as I mentioned.)

The plus point is precisely that it's language-agnostic. Writing a wrapper for the language you use is relatively trivial.

This is pretty much what resource compilers were doing on Windows and MacOS. But this approach only requires binutils and is thus portable as long as you are using binutils.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 15, 2023, 08:25:26 pm
So injecting binary static data in this manner is deemed sufficient? an ability have a compiler do this simply by compiling the code, wouldn't be that helpful?

It's not "injecting" data any more than a compiler injects data with compiled object code. It's the same thing.

I actually kinda like it. It's generating object files from binary files, adding this in the programming language itself doesn't add anything much, except possibly making it cuter, but this can be made cuter by wraoping it with a few lines of code (as I mentioned.)

The plus point is precisely that it's language-agnostic. Writing a wrapper for the language you use is relatively trivial.

This is pretty much what resource compilers were doing on Windows and MacOS. But this approach only requires binutils and is thus portable as long as you are using binutils.

Well this is precisely the kind of thing that I want to support, its easy to implement, like the switchable keyword lexicons, I need to think about it though, step back and see the 'big picture', the preprocessor is all about executing code at compile time and I want to take a long look at preprocessing anyway.


Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 15, 2023, 08:29:16 pm
Here is some test source code used to help me test the grammar of the new language. This is just to give an idea of how code would look once one started to use it. You'll see a bunch of things in here, using @ for labels and singly/doubly quoted string literals that can contain quoted substrings and span multiple lines, also you'll see subscripted labels too. Most of the comments are fictitious, just to give the code a real world appearance.

Code: [Select]
procedure main (arg);

    /* This is sample code */

    dcl name string(32);
    dcl I bin(15);
    dcl J bin(15);
    dcl S bin(31);
    dcl A(1024) bit(8); // 1 KByte
    dcl root ptr;
    dcl status bin(15);

    type_a = "string literlas can be single quoted like this";

    type_b = ""or double quoted like this, with embedded single quoted text "like this" see?"";

    name = ""demonstration_network_controller"" ;

    title = ""This is the "best in class"
   
    language system"" ;

    //call sys$announce_device(name, status);

    if status ~= 0 then
       return;
    end;

    call sys$install_platform(title);

    root = sys$get_system_bootroot(name);

    call sys$validate_authority(root);

    goto cycle_loop;

@cycle_loop

    I = 100;

    if I > 0 then
       goto cycle_loop;
    end;
   
    I = sys$get_updated_count(J);

@setpoint(0)

    loop while (I >= J)
       I = get_updated_count(J);
    end;

    if I = 123 & J = 321 then
       I = 0;
       goto setpoint(I); // never pass non-zero !!
    elif J + I > J * I then
       goto cycle_loop;
    end;

    call get_latest_faulting_stack(S);

    if S ~= 0 then
       call sys$stack_crawler(S);
    end;

    /* set S to the sentinel for the next operation */

    S = F5D3 03A2:H; // we need to ensure this sentinel is not allocated to any other device types

    call sys$reinitialize_firmware_table(S);

    call sys$sleep (1024);

    goto cycle_loop ;

/* Crawl the stack chain looking for the designated handler */
procedure sys$stack_crawler (handler_id) recursive;

    arg handler_id bin(31);

end;

/* Only call this if we kmow there's no active services still running */
procedure sys$reinitialize_firmware_table(table);

    arg table bin(31);

end;

procedure sys$timer_callback (context) interrupt;

    arg context pointer;

end;

end;


Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 15, 2023, 08:46:54 pm
So injecting binary static data in this manner is deemed sufficient? an ability have a compiler do this simply by compiling the code, wouldn't be that helpful?

It's not "injecting" data any more than a compiler injects data with compiled object code. It's the same thing.

I actually kinda like it. It's generating object files from binary files, adding this in the programming language itself doesn't add anything much, except possibly making it cuter, but this can be made cuter by wraoping it with a few lines of code (as I mentioned.)

The plus point is precisely that it's language-agnostic. Writing a wrapper for the language you use is relatively trivial.

This is pretty much what resource compilers were doing on Windows and MacOS. But this approach only requires binutils and is thus portable as long as you are using binutils.

Well this is precisely the kind of thing that I want to support, its easy to implement, like the switchable keyword lexicons, I need to think about it though, step back and see the 'big picture', the preprocessor is all about executing code at compile time and I want to take a long look at preprocessing anyway.

Well, the "wrapper" I was talking about uses some preprocessor to help, so if you implement a better preprocessor than the simple C one, that sure would help here.

Just be aware that writing good compile-time stuff that can fully replace a preprocessor (and do more and better) is no simple task.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 15, 2023, 08:58:39 pm
So injecting binary static data in this manner is deemed sufficient? an ability have a compiler do this simply by compiling the code, wouldn't be that helpful?

It's not "injecting" data any more than a compiler injects data with compiled object code. It's the same thing.

I actually kinda like it. It's generating object files from binary files, adding this in the programming language itself doesn't add anything much, except possibly making it cuter, but this can be made cuter by wraoping it with a few lines of code (as I mentioned.)

The plus point is precisely that it's language-agnostic. Writing a wrapper for the language you use is relatively trivial.

This is pretty much what resource compilers were doing on Windows and MacOS. But this approach only requires binutils and is thus portable as long as you are using binutils.

Well this is precisely the kind of thing that I want to support, its easy to implement, like the switchable keyword lexicons, I need to think about it though, step back and see the 'big picture', the preprocessor is all about executing code at compile time and I want to take a long look at preprocessing anyway.

Well, the "wrapper" I was talking about uses some preprocessor to help, so if you implement a better preprocessor than the simple C one, that sure would help here.

Just be aware that writing good compile-time stuff that can fully replace a preprocessor (and do more and better) is no simple task.

Yes it isn't simple, Antlr4 however is extremely powerful, it makes it easy to experiment and iterate the grammar rules without writing a line of code. The way to do the preprocessor too would be to devise a separate grammar for the preprocessor language, then that would become an initial phase, consume source and perform preprocessing then, the output from that gets parsed as a phase 2, using the actual language grammar. Once again I'll be studying PL/I's preprocessor, I hardly used it when I used to write PL/I but apparently it was very powerful supporting meta-loops, meta-ifs and even meta-procedures, there may be a truly useful core that I can "borrow".

See page 699 of this doc (http://publibfp.boulder.ibm.com/epubs/pdf/ibm3lr80.pdf) if you're interested in the PL/I preprocessor.

C's #define is based on PL/I's %replace and C's #include is based on PL/I's %include.

Here's an example that expresses a loop as a preprocessor loop, the loop is replaced by multiple distinct assignment statements by the preprocessor:

[attachimg=1]

This does something similar (it seems) the stuff that the Zig preprocessor can do.


Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 15, 2023, 09:36:27 pm
Pure garbage. Again.
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 15, 2023, 09:48:23 pm
@SiliconWizard
"But this approach only requires binutils "

Yup, not supported by Acorn DSE (riscOS), CodeWarrior-SONY-ps1 (Windows) and Sierra (dos).

No binutils -> you need a specific tool.

The Nintendo sdk for GB and GBA come with a tool to convert a gif image into a c file stuffing an array with hex values.
The Acorn DSE does the same.
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 15, 2023, 09:55:22 pm
@SiliconWizard
"But this approach only requires binutils "

Yup, not supported by Acorn DSE (riscOS), CodeWarrior-SONY-ps1 (Windows) and Sierra (dos).

No binutils -> you need a specific tool.

The Nintendo sdk for GB and GBA come with a tool to convert a gif image into a c file stuffing an array with hex values.
The Acorn DSE does the same.

As long as your tools use a separate linker and object files, then it's usually doable as with binutils one way or another - you may have to write your own utility to generate an object file from binary data if it doesn't exist already.
Now if you use a compiler that has an integrated linker and gives you no easy access to intermediate object files, then obviously your only bet is to stuff data in variables directly in the programming language itself.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 15, 2023, 10:09:24 pm
Pure garbage. Again.

Perhaps programming language design isn't a good fit for you, you feel inadequate, unable to contribute meaningfully, so you lash out, your emotions take over.

You might do better with flower arranging or something less intellectually strenuous, a man must know his limitation after all...
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 15, 2023, 10:58:52 pm
Pure garbage. Again.

Perhaps programming language design isn't a good fit for you, you feel inadequate, unable to contribute meaningfully, so you lash out, your emotions take over.

You might do better with flower arranging or something less intellectually strenuous, a man must know his limitation after all...


 :-DD :-DD :-DD :-DD
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 15, 2023, 11:04:06 pm
@guys, ignore this troll, seriously, and he will leave the forum.
Otherwise we have to babysit this grown dude.
I am seriously going to ignore the whole topic, I have seriously enough.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 15, 2023, 11:12:07 pm
@guys, ignore this troll, seriously, and he will leave the forum.
Otherwise we have to babysit this grown dude.
I am seriously going to ignore the whole topic, I have seriously enough.

At last, no more juvenile posts and outbursts!

If the subject doesn't interest you or you feel you can't positively contribute then you SHOULD leave the thread and everyone else here, alone, what harm is being done? what are you afraid of? why be a pest?

The way you whine here you'd think I was sending you personal message or spam or making harassing phone calls, the fact is the thread is just a discussion nobody asked you for anything, you are clearly the the troll here FFS.

There's always one, someone who wants to bicker and insult and offend, it's sad, if you don't want to discuss the subject then don't what's hard to understand here?

Title: Re: The Imperium programming language - IPL
Post by: EEVblog on January 16, 2023, 01:24:19 am
Both of you, please quit the bickering. Either contribute to the thread or ignore it and/or ignore each other.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 16, 2023, 07:28:17 pm
I've drafted a document that compares language features of C, PL/I and the hypothetical IPL, where the features are conducive to writing an operating system in that language, for those interested jut visit the page in GitHub (https://github.com/Steadsoft/imperium/blob/main/docs/README.md).

This arose from discussions elsewhere. The impetus was my view that a language good for MCU work must also be suitable for developing an operating system, any language claiming to be "good" at hardware programming must also be good for OS development.

To my knowledge few language have a track record of delivering real operating systems, PL/I and C are the most well known I think.



Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 16, 2023, 07:56:59 pm
I've drafted a document that compares language features of C, PL/I and the hypothetical IPL, where the features are conducive to writing an operating system in that language, for those interested jut visit the page in GitHub (https://github.com/Steadsoft/imperium/blob/main/docs/README.md).

This arose from discussions elsewhere. The impetus was my view that a language good for MCU work must also be suitable for developing an operating system, any language claiming to be "good" at hardware programming must also be good for OS development.

To my knowledge few language have a track record of delivering real operating systems, PL/I and C are the most well known I think.

Algol is another such language. It continues to be commercially successful in delivering operating systems.

xC is another for MCUs.

Smalltalk and Squeak/Pharo are another, and have been at least as influential as C, albeit indirectly. I'll bet that in the long term Smalltalk will be more influential than C.

You would benefit from having a wider experience and less of a tunnel vision.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 16, 2023, 08:05:26 pm
BTW, I hate the quotation ("When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.") in your .sig with a vengence. It implies that the person really can imagine all possibilities, thus precluding genuine step advances.

If I was to say "I can't imagine how spooky action at a distance can work, therefore it can be discounted", then I would be correctly vilified, and/or be a laughing stock, and/or ignored.
Title: Re: The Imperium programming language - IPL
Post by: cfbsoftware on January 16, 2023, 09:11:29 pm
To my knowledge few language have a track record of delivering real operating systems
Here are some such languages I can think of:

Pascal:
  UCSD Pascal and CP/M-68K

Modula-2: 
  Acorn Archimedes and ETH Medos-2 

Oberon:
  Project Oberon, ETHOS, AOS/A2

Also read the article in The Register: The wild world of non-C operating systems

https://www.theregister.com/2022/03/29/non_c_operating_systems/ (https://www.theregister.com/2022/03/29/non_c_operating_systems/)
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 16, 2023, 09:14:56 pm
I've drafted a document that compares language features of C, PL/I and the hypothetical IPL, where the features are conducive to writing an operating system in that language, for those interested jut visit the page in GitHub (https://github.com/Steadsoft/imperium/blob/main/docs/README.md).

This arose from discussions elsewhere. The impetus was my view that a language good for MCU work must also be suitable for developing an operating system, any language claiming to be "good" at hardware programming must also be good for OS development.

To my knowledge few language have a track record of delivering real operating systems, PL/I and C are the most well known I think.

<Waffle, Waffle, Waffle>

You would benefit from having a wider experience and less of a tunnel vision.

So would you.



Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 16, 2023, 09:16:24 pm
To my knowledge few language have a track record of delivering real operating systems
Here are some such languages I can think of:

Pascal:
  UCSD Pascal and CP/M-68K

Modula-2: 
  Acorn Archimedes and ETH Medos-2 

Oberon:
  Project Oberon, ETHOS, AOS/A2

Also read the article in The Register: The wild world of non-C operating systems

https://www.theregister.com/2022/03/29/non_c_operating_systems/ (https://www.theregister.com/2022/03/29/non_c_operating_systems/)

Thanks, I did however say "few languages" there are just a few that's the point I was making, its like six or seven languages it seems.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 16, 2023, 09:21:09 pm
My core thesis here though is that if a language is ill suited to writing an OS then it's ill suited for use as an MCU language, I'm using that as a kind of criteria, what would one expect from a language if they wanted to write an OS? those same capabilities would be huge when using the language to code for MCUs.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 16, 2023, 10:19:07 pm
Operator symbols

Given that I want to support arithmetic/logical shift left/right along with rotate left and right, the question emerges as to what priorities these operators should be given as well as the actual symbol to use for them.

For rotation these sprang to mind:

Rotate left: <@<
Rotate right: >@>

The circularity of the @ seems a reasonable sign for rotation.

Anyway, suggestions welcome.

Title: Re: The Imperium programming language - IPL
Post by: gf on January 16, 2023, 10:25:12 pm
My core thesis here though is that if a language is ill suited to writing an OS...

In a completely abstract, machine-independent way?
With or without a certain amount of additional assembly code?
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 16, 2023, 10:28:33 pm
My core thesis here though is that if a language is ill suited to writing an OS then it's ill suited for use as an MCU language,

Well, I think almost everyone says that, even these days, although it's usually defined in the positive form rather than the negative one that you used here.
It's what is commonly called a "system programming language". A reasonable list is there:
https://en.wikipedia.org/wiki/System_programming_language

It's not exhaustive though by any means. Just regarding Wirth languages, they don't even list Oberon, while it's been designed (and used) to develop a full OS, and it's being used as a language for programming MCUs (which is the whole business of one of the forum users here.)
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 16, 2023, 10:48:51 pm
My core thesis here though is that if a language is ill suited to writing an OS...

In a completely abstract, machine-independent way?

With or without a certain amount of additional assembly code?

To the most reasonable extent possible yes. The changes we'd need to make to the OS codebase if we moved from one target to another, should be as small as we can reasonably make it. Ideally we'd "just" recompile the bulk of the source for the new target and we're good. I don't seek to eliminate assembler, at least that's not a goal, having a small core of assembly language bootstrap code etc is not a big deal, seems reasonable.

If you had to write a serious OS like Windows, Linux, Android etc, what essential language features would you need, if there were umpteen potentially suitable language available, how would you choose (ignoring things like skills availability, popularity etc).

I guess "portability" is an obvious one I need to add to my list, although the term itself needs some kind of definition. But reducing unpredictability is very high on the list, semantic behavior that is left to the implementor or is somehow undefined should be kept to an absolute minimum, stuff like the way C has an undefined order of evaluation when passing args and so on.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 16, 2023, 10:53:02 pm
My core thesis here though is that if a language is ill suited to writing an OS then it's ill suited for use as an MCU language,

Well, I think almost everyone says that, even these days, although it's usually defined in the positive form rather than the negative one that you used here.
It's what is commonly called a "system programming language". A reasonable list is there:
https://en.wikipedia.org/wiki/System_programming_language

It's not exhaustive though by any means. Just regarding Wirth languages, they don't even list Oberon, while it's been designed (and used) to develop a full OS, and it's being used as a language for programming MCUs (which is the whole business of one of the forum users here.)

Right, so what makes a language a "systems programming language"? The Wikipedia article lists a bunch but those are systems programming language by virtue of the fact they've been used to do "systems programming", I'm curious about what specifically must a language have to "be" such a language.

Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?
Title: Re: The Imperium programming language - IPL
Post by: westfw on January 16, 2023, 10:54:32 pm
Quote
My core thesis here though is that if a language is ill suited to writing an OS then it's ill suited for use as an MCU language
I dunno.  Any language can write an OS, given appropriate "support functions" written in some other language.

Are you really counting early CP/M and MSDOS 1 as "operating systems"?  That's a pretty low bar.

BTW, CP/M 2.2 (which was what was heavily used) appears to have been re-written in assembly language. http://www.cpm.z80.de/source.html (http://www.cpm.z80.de/source.html)

What about "self-hosting"?  Is it important for your language to be able to compile itself?  This seems to be an implicit requirement for modern languages, but I'm not really sure it should be.

Finally, I'm not sure what the hangup is about avoiding assembly language is, in the first place.  By the time you talk about some of the features that have come up ("equal cycle length code paths"), I think you've already thrown portability - a major selling point of HLLs - out the door...

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 16, 2023, 10:55:23 pm
For an OS too I'm including device driver code, code like we see on MCUs, that must interface to very specific aspects of buses and device protocols and so on.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 16, 2023, 11:02:57 pm
Quote
My core thesis here though is that if a language is ill suited to writing an OS then it's ill suited for use as an MCU language
I dunno.  Any language can write an OS, given appropriate "support functions" written in some other language.

Are you really counting early CP/M and MSDOS 1 as "operating systems"?  That's a pretty low bar.

BTW, CP/M 2.2 (which was what was heavily used) appears to have been re-written in assembly language. http://www.cpm.z80.de/source.html (http://www.cpm.z80.de/source.html)

What about "self-hosting"?  Is it important for your language to be able to compile itself?  This seems to be an implicit requirement for modern languages, but I'm not really sure it should be.

Finally, I'm not sure what the hangup is about avoiding assembly language is, in the first place.  By the time you talk about some of the features that have come up ("equal cycle length code paths"), I think you've already thrown portability - a major selling point of HLLs - out the door...

Interesting points. Sure CP/M was (at the time anyway) an OS, Kildall crafted most of it in PL/I too, they were simple by todays standards but they managed to write a lot of code not in assembler. As for "self hosting" or "boot strapping" that's a great way to harden a language, compiler but not that important. I'd write a new language in C# for example, a means to an end. I think bootstrapping was important in the 60s because there was no other language around to write the compiler, so it was assembler first then some version 0.0001 of the language used to rewrite that assembler and so on, the Multics guys did that in fact initially.

Portability? well yes, but we would want to minimize, isolate as much as we can of that non-portability.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 16, 2023, 11:14:54 pm
If we look at "modern" C there are a long list of things that have been added, some in recent standards and others as vendor extensions, so the way I am thinking of this is to devise a new language with all of these historic changes and extensions as hindsight, stuff one can incorporate from the outset rather than forty years later.

The grammar I've selected (basically derived from PL/I) is not selected for systems programming reasons but primarily for lacking reserved words, letting new keywords appear over time with ZERO impact on existing source code. If that requirement is not met then we get silliness - eventually - like this post (https://blog.paranoidcoding.com/2022/04/11/lowercase-type-names.html) describes.

That's pretty fundamental, all of a sudden we can't use lowercase names for classes, structs? This is a modern version of rules like all labels must begin column 2 and so on!





Title: Re: The Imperium programming language - IPL
Post by: westfw on January 17, 2023, 07:42:19 am
Re: binary file inclusion...
Quote
Code: [Select]
LIBS += -Wl,--format=binary -Wl,--start-group $(RESOURCES) -Wl,--end-group -Wl,--format=default

The gnu binutils "objcopy" utility also gives you the option of creating a linkable (.elf format) file from arbitrary binary data, and produces the same convenient symbols (I imagine it shares binary input conversion code with the linker):

Code: [Select]
> avr-objcopy -O elf32-avr -I binary binfile foo.elf
Code: [Select]
> avr-objdump -x foo.elf
Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .data         0000000b  00000000  00000000  00000034  2**0
                  CONTENTS, ALLOC, LOAD, DATA
SYMBOL TABLE:
00000000 l    d  .data  00000000 .data
00000000 g       .data  00000000 _binary_binfile_start
0000000b g       .data  00000000 _binary_binfile_end
0000000b g       *ABS*  00000000 _binary_binfile_size

I don't know whether I consider that more, or less, convenient than the linker commands; it gives you the opportunity to make an ordinary makefile rule for "compiling" .png files (or whatever) to .o files, which can then also be linked normally.

I about died when I discovered that a team at work was using the whole "convert to a C source file containing a big char array" method on a several-hundred megabyte linux .iso image :-(



Title: Re: The Imperium programming language - IPL
Post by: MIS42N on January 17, 2023, 10:11:45 am
Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?
I think it was the concept of a pointer *, which often resolved to a memory address, and that address could be just about anything. Fun things like **thing were simple to write (but maybe not so simple to understand). Also the inclusion of all bit operators and/or/xor/shifts. The standard library includes system type functions like malloc(). A lot of C maps well to machine instructions. And the lack of any nanny code like array bound checking made it fast (but dangerous).

It persists because it is still dangerous. Many languages try to avoid the danger and impose rules that are 95% helpful and 5% frustrating.

In my opinion.

Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 17, 2023, 10:48:16 am
Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?
I think it was the concept of a pointer *, which often resolved to a memory address, and that address could be just about anything. Fun things like **thing were simple to write (but maybe not so simple to understand). Also the inclusion of all bit operators and/or/xor/shifts. The standard library includes system type functions like malloc(). A lot of C maps well to machine instructions. And the lack of any nanny code like array bound checking made it fast (but dangerous).

It persists because it is still dangerous. Many languages try to avoid the danger and impose rules that are 95% helpful and 5% frustrating.

In my opinion.

Well, pointers were there in languages that influenced the design of C.

To take a less well known example, consider Algol68. INT x states x is an immutable integer, REF INT x states x is a reference to an integer which is variable, REF  REF INT state x is a pointer to an integer. In Algol 68 the types [10]INT, [10]REF INT, REF[10]INT, REF[10]REF INT are all distinct types and all useful. They closely correspond to the types declared by these C declarations:
typedef const int i[10]; typedef const int *ir[10]; typedef int const *ri[10]; typedef int const *rir[10]; http://www.cap-lore.com/Languages/aref.html (http://www.cap-lore.com/Languages/aref.html)

I wonder if the OP realises that Algol68 has many of the features he deems desirable, e.g. "if" can be a variable, and it has symbol set (with alternatives) that work on the various different character sets available on computers at the time. And those are just the start. https://opensource.com/article/20/6/algol68 (https://opensource.com/article/20/6/algol68) Truly, Algol is an improvement on most of its successors.

I suspect the OP is unaware of https://rosettacode.org/wiki/Rosetta_Code (https://rosettacode.org/wiki/Rosetta_Code) which gives the same program in many different languages. Examples: there are 68 implementations of Dijkstra's algorithms, and bitwise operations implemented in 144 languages.

That really ought to give him something to "compare and contrast" before committing to something for his language.
Title: Re: The Imperium programming language - IPL
Post by: MIS42N on January 17, 2023, 11:15:06 am
Well, pointers were there in languages that influenced the design of C.
I was not intending to imply C was the first language to use pointers. What I was trying to say is the confluence of pointers, all the logical operators, and a standard library that has useful system oriented functions gives C the right ingredients to be a systems programming language. Which was what OP asked.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 17, 2023, 11:54:19 am
Well, pointers were there in languages that influenced the design of C.
I was not intending to imply C was the first language to use pointers. What I was trying to say is the confluence of pointers, all the logical operators, and a standard library that has useful system oriented functions gives C the right ingredients to be a systems programming language. Which was what OP asked.

True, but they weren't unique to C.

What was unique to C was that it came as part of a zero cost operating system available on the minicomputers that were becoming widely available.

Get UNIX for cheapness and experimentation, get a free high (for the time) level language. All important for universities, research labs, and medium sized companies. What was in C was more or less irrelevant; all languages had problems and everybody found ways of working around them

Unfortunately current computers are nothing like PDP11s
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 17, 2023, 02:26:36 pm
Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?
I think it was the concept of a pointer *, which often resolved to a memory address, and that address could be just about anything. Fun things like **thing were simple to write (but maybe not so simple to understand). Also the inclusion of all bit operators and/or/xor/shifts. The standard library includes system type functions like malloc(). A lot of C maps well to machine instructions. And the lack of any nanny code like array bound checking made it fast (but dangerous).

It persists because it is still dangerous. Many languages try to avoid the danger and impose rules that are 95% helpful and 5% frustrating.

In my opinion.

Well, pointers were there in languages that influenced the design of C.

To take a less well known example, consider Algol68. INT x states x is an immutable integer, REF INT x states x is a reference to an integer which is variable, REF  REF INT state x is a pointer to an integer. In Algol 68 the types [10]INT, [10]REF INT, REF[10]INT, REF[10]REF INT are all distinct types and all useful. They closely correspond to the types declared by these C declarations:
typedef const int i[10]; typedef const int *ir[10]; typedef int const *ri[10]; typedef int const *rir[10]; http://www.cap-lore.com/Languages/aref.html (http://www.cap-lore.com/Languages/aref.html)

I wonder if the OP realises that Algol68 has many of the features he deems desirable, e.g. "if" can be a variable, and it has symbol set (with alternatives) that work on the various different character sets available on computers at the time. And those are just the start. https://opensource.com/article/20/6/algol68 (https://opensource.com/article/20/6/algol68) Truly, Algol is an improvement on most of its successors.

I suspect the OP is unaware of https://rosettacode.org/wiki/Rosetta_Code (https://rosettacode.org/wiki/Rosetta_Code) which gives the same program in many different languages. Examples: there are 68 implementations of Dijkstra's algorithms, and bitwise operations implemented in 144 languages.

That really ought to give him something to "compare and contrast" before committing to something for his language.

Algol68 was the language that most influenced PL/I which I've spoken about at length here, there was also a build of it that accepted Russian keywords, I wonder if you were aware of that.

I have to ask, politely, would you please stop insinuating that I am "unqualified" in some way or other, to discuss this subject? Repeatedly making disparaging remarks and insulting comments is really not what I expect in any forum like this.

If you cannot disagree with someone without implying they are inferior, then there's really no point in discussing anything with anyone.

I went to the trouble in my very first post to mention that I was experienced with compiler design and development, I anticipated that some might be skeptical, that a language is a large undertaking and not to be treated glibly, I even pointed to the source code for the entire system (https://github.com/Steadsoft/PLI-2000). I know all that and therefore summarized what I had done before, yet you speak to me as if I'm a dimwitt, well I have the experience buddy, I know of what I speak, I know how large a project this can be, I know what's involved, I'm likely far more able to do this than you.

I've attached an example of the assembler code listing generated by that compiler for the 32 bit Intel CPU, tell me, how does that not carry any weight with you? The compiler did not simply generate assembler, it generated COFF object and dll files, getting assembler was a compile option, it generated that listing as it generated the code (https://github.com/Steadsoft/PLI-2000/blob/1a711f694572de36bbdf74b65b1471cc735ce48c/Compiler%20Source/EMITTER.C#L1463).

So I've actually done this before, thirty years ago buddy, the whole shebang, when there was no internet, no tools like Antlr, and I did most of it with Borland C on DOS with a VGA screen, have you ever built anything comparable from the ground up? what makes you such an expert that you feel you can tell me how to do things?

[attachimg=1]


Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 17, 2023, 03:28:32 pm
Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?
I think it was the concept of a pointer *, which often resolved to a memory address, and that address could be just about anything. Fun things like **thing were simple to write (but maybe not so simple to understand). Also the inclusion of all bit operators and/or/xor/shifts. The standard library includes system type functions like malloc(). A lot of C maps well to machine instructions. And the lack of any nanny code like array bound checking made it fast (but dangerous).

It persists because it is still dangerous. Many languages try to avoid the danger and impose rules that are 95% helpful and 5% frustrating.

In my opinion.

So this potential new language will have pointers but also offsets. That is we'll be able to manipulate the address of a datum and also the offset of a datum relative to some base. This is a powerful feature lacking in C.

Changing the subject for a moment I once designed a fast in memory database technology that ran on Windows. This leveraged memory mapped files, so a "database" was in fact a snapshot of a portion of process address space, say 16GB for args sake.

That memory consisted of a tree of data structures, quite complex with index trees and all kinds of data with pointers to other data all situated within that 16GB contiguous block.

But any process could map that file into any part of its address space (this was shared memory, multiple processes could access it concurrently), the OS decides where to map such data into the virtual address space as and when a process maps it and that actual address within each process can vary (one can force it to map at some specific address but that's hardly a good idea).

Therefore in the data the links could not be stored as pointers and referenced and dereferenced as pointers, the tree nodes had to store an offset (from the start address of the 16GB block).

Then at runtime (using C macros) we referenced and deference using the offsets, the macro did the arithmetic to compute the actual process relative pointer).

If C had offered me the ability to use language supported offsets directly, all of that plumbing would vanish, and the cost of calculating pointers/offsets would have all but vanished, being done wholly in the addressing code generated by the compiler.

PL/I always offered that and the example I give, although not directly applicable to an MCU, is a real world example of how such a feature is valuable.

So we'll be able to write code that can load a linked list into memory (perhaps from disk or somewhere) and have that linked list 100% relocatable, the actual address that it gets put is immaterial because no absolute pointers are stored only relative offsets, the notation used in the code "->" is the same, whether we're referring to a pointer or an offset, the language does the work.

We'd be able to save/load self referential data structures that can be loaded at any address and still work fine, that's quite a neat feature.

That's the kind of systems programming capability I'm talking about here.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 17, 2023, 07:17:14 pm
Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?
I think it was the concept of a pointer *, which often resolved to a memory address, and that address could be just about anything. Fun things like **thing were simple to write (but maybe not so simple to understand). Also the inclusion of all bit operators and/or/xor/shifts. The standard library includes system type functions like malloc(). A lot of C maps well to machine instructions. And the lack of any nanny code like array bound checking made it fast (but dangerous).

It persists because it is still dangerous. Many languages try to avoid the danger and impose rules that are 95% helpful and 5% frustrating.

In my opinion.

Well, pointers were there in languages that influenced the design of C.

To take a less well known example, consider Algol68. INT x states x is an immutable integer, REF INT x states x is a reference to an integer which is variable, REF  REF INT state x is a pointer to an integer. In Algol 68 the types [10]INT, [10]REF INT, REF[10]INT, REF[10]REF INT are all distinct types and all useful. They closely correspond to the types declared by these C declarations:
typedef const int i[10]; typedef const int *ir[10]; typedef int const *ri[10]; typedef int const *rir[10]; http://www.cap-lore.com/Languages/aref.html (http://www.cap-lore.com/Languages/aref.html)

I wonder if the OP realises that Algol68 has many of the features he deems desirable, e.g. "if" can be a variable, and it has symbol set (with alternatives) that work on the various different character sets available on computers at the time. And those are just the start. https://opensource.com/article/20/6/algol68 (https://opensource.com/article/20/6/algol68) Truly, Algol is an improvement on most of its successors.

I suspect the OP is unaware of https://rosettacode.org/wiki/Rosetta_Code (https://rosettacode.org/wiki/Rosetta_Code) which gives the same program in many different languages. Examples: there are 68 implementations of Dijkstra's algorithms, and bitwise operations implemented in 144 languages.

That really ought to give him something to "compare and contrast" before committing to something for his language.

Algol68 was the language that most influenced PL/I which I've spoken about at length here, there was also a build of it that accepted Russian keywords, I wonder if you were aware of that.

I was aware of the Russian Algol68 variant thanks. PL/1 never made an impression on this side of the pond. The nearest was a brief flurry of interest in PL/M in the late 70s.

You have seemed to be interested in expressing bit operations. Which of the 144 examples are you thinking of following. Do you prefer the VHDL or Smalltalk concepts? If none of the 144, what will be in your 145th variant?

Quote
I have to ask, politely, would you please stop insinuating that I am "unqualified" in some way or other, to discuss this subject? Repeatedly making disparaging remarks and insulting comments is really not what I expect in any forum like this.

It isn't necessarily disparaging to regard someone as unqualified; we have all been unqualified. The key point is what any such person does or does not do to become more qualified. "Learning the literature/history" is necessary.

Quote
I went to the trouble in my very first post to mention that I was experienced with compiler design and development, I anticipated that some might be skeptical, that a language is a large undertaking and not to be treated glibly, I know all that and therefore summarized what I had done before.

Yet you continue to insinuate I'm some kind of idiot.

Implementing a compiler and selecting which concepts to use/discard when creating a language have very little in common.

From your statements in this thread, it appears that your experience is limited to a relatively narrow subset of the conceptual types of languages in existence[1]. You fail to engage with suggestions that would widen (possibly beneficially) your understanding. For example, you've completely ignored the possibility of your language including one facility that is very useful in resource-constrained embedded systems: fixed point arithmetic. I don't remember you discussing how multicore/multithread parallel processing will be expressed in your language; that will be critically important in the future.

I try to become less inexperienced in the things that matter to me. Knowing how and when to use languages that are fundamentally and conceptually very different is one such case. OTOH, implementing a compiler is completely uninteresting to me.

[1] e.g. C, Pascal, PL/M, Delphi, Coral-66 are all procedural languages with little difference between them. Conceptually very different languages are VHDL, LISP, Smalltalk, Forth, OCCAM/xC etc etc.
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 17, 2023, 07:33:37 pm
Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?

I don't get the premise of your question nor the question itself. It looks like some users that have replied to this have taken it as "what are the features of C that make it appropriate as a system programming language", but that is not what I read in the above, that's not what you asked.

The exact answer to your exact question is obvious - C was explicitely designed to implement an OS. It doesn't need a "status", it's just what it was meant for. And history has proven a long track record of it being appropriate indeed for OS development and low-level stuff in general.

That doesn't mean it's "perfect", or that it's the only language for such endeavours. Other languages have been clearly used, as we already mentioned. A good chunk of the initial MacOS, and base applications was written in Pascal. Wirth designed Oberon as a language for implementing the Oberon OS. Ada has been extensively used for embedded development, and there are a few OSs written in Ada as well.

If the question was "what features of C are key features for a system prog language", then the answer is relatively simple. It has high-level features on par with languages such as Pascal (so not very high-level, bug high-level enough to express abstraction), but can get low-level enough (yes, in particular through the use of pointers, pointer arithmetic, logic operators, function pointers, ...) to address hardware almost directly.

Any modern language that wants to do the same usually resorts to "unsafe" blocks of code, such as Rust, but the idea of unsafe code was already there in Modula-3 - I suggested reading the language report which can be found online. The point is that once you have to use your language for system programming, you'll have to add ways of circumventing all your nice protections by adding "unsafe"constructs.

While that may look like a reasonable approach - instruct the developers to use safe code as much as possible, and restrict unsafe code to the bare minimum - the same can be achieved using multiple languages. Using a very high-level language along with assembly, or along with C for the very low-level code, also works, and is not significantly better or worse than trying to stuff all source code with a single language and resorting to tricks to achieve that.

Just my 2 cents.

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 17, 2023, 08:28:35 pm
Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?
I think it was the concept of a pointer *, which often resolved to a memory address, and that address could be just about anything. Fun things like **thing were simple to write (but maybe not so simple to understand). Also the inclusion of all bit operators and/or/xor/shifts. The standard library includes system type functions like malloc(). A lot of C maps well to machine instructions. And the lack of any nanny code like array bound checking made it fast (but dangerous).

It persists because it is still dangerous. Many languages try to avoid the danger and impose rules that are 95% helpful and 5% frustrating.

In my opinion.

Well, pointers were there in languages that influenced the design of C.

To take a less well known example, consider Algol68. INT x states x is an immutable integer, REF INT x states x is a reference to an integer which is variable, REF  REF INT state x is a pointer to an integer. In Algol 68 the types [10]INT, [10]REF INT, REF[10]INT, REF[10]REF INT are all distinct types and all useful. They closely correspond to the types declared by these C declarations:
typedef const int i[10]; typedef const int *ir[10]; typedef int const *ri[10]; typedef int const *rir[10]; http://www.cap-lore.com/Languages/aref.html (http://www.cap-lore.com/Languages/aref.html)

I wonder if the OP realises that Algol68 has many of the features he deems desirable, e.g. "if" can be a variable, and it has symbol set (with alternatives) that work on the various different character sets available on computers at the time. And those are just the start. https://opensource.com/article/20/6/algol68 (https://opensource.com/article/20/6/algol68) Truly, Algol is an improvement on most of its successors.

I suspect the OP is unaware of https://rosettacode.org/wiki/Rosetta_Code (https://rosettacode.org/wiki/Rosetta_Code) which gives the same program in many different languages. Examples: there are 68 implementations of Dijkstra's algorithms, and bitwise operations implemented in 144 languages.

That really ought to give him something to "compare and contrast" before committing to something for his language.

Algol68 was the language that most influenced PL/I which I've spoken about at length here, there was also a build of it that accepted Russian keywords, I wonder if you were aware of that.

I was aware of the Russian Algol68 variant thanks. PL/1 never made an impression on this side of the pond. The nearest was a brief flurry of interest in PL/M in the late 70s.

Are you claiming there were no computer installations in the UK that used PL/I?

You have seemed to be interested in expressing bit operations. Which of the 144 examples are you thinking of following. Do you prefer the VHDL or Smalltalk concepts? If none of the 144, what will be in your 145th variant?

I'm considering symbolic operators for left/right logical shift, right arithmetic shift, rotate left and rotate right, and possibly population count, is that what you're asking me?

Quote
I have to ask, politely, would you please stop insinuating that I am "unqualified" in some way or other, to discuss this subject? Repeatedly making disparaging remarks and insulting comments is really not what I expect in any forum like this.

It isn't necessarily disparaging to regard someone as unqualified; we have all been unqualified. The key point is what any such person does or does not do to become more qualified. "Learning the literature/history" is necessary.

The key point is that you are conflating (no doubt intentionally) one's certifications with one's argument. You attack a person's arguments or ideas not by counter arguments but by insults and demeaning comments. One doesn't attack or undermine an argument by attacking or undermining the person presenting the argument. This is a common fallacy committed by inexperienced or disingenuous debaters, those who prefer rhetoric to logic.

If I made some claim to a mathematician, developed some proof of some conjecture, the mathematician would never taint their evaluation of said proof by judging my personality, qualifications, what literature I might have read, what experiences I might have had, he'd argue, respond, on the basis of facts and logic, consider this from Noam Chomsky:

Quote from: Noam Chomsky
In my own professional work I have touched on a variety of different fields. I've done my work in mathematical linguistics, for example, without any professional credentials in mathematics; in this subject I am completely self-taught, and not very well taught. But I've often been invited by universities to speak on mathematical linguistics at mathematics seminars and colloquia. No one has ever asked me whether I have the appropriate credentials to speak on these subjects; the mathematicians couldn't care less. What they want to know is what I have to say. No one has ever objected to my right to speak, asking whether I have a doctor's degree in mathematics, or whether I have taken advanced courses in the subject. That would never have entered their minds. They want to know whether I am right or wrong, whether the subject is interesting or not, whether better approaches are possible—the discussion dealt with the subject, not with my right to discuss it.

Emphasis mine.

You might want to give his words some serious thought, perhaps get more familiar with the literature as it were.

Quote
I went to the trouble in my very first post to mention that I was experienced with compiler design and development, I anticipated that some might be skeptical, that a language is a large undertaking and not to be treated glibly, I know all that and therefore summarized what I had done before.

Yet you continue to insinuate I'm some kind of idiot.

Implementing a compiler and selecting which concepts to use/discard when creating a language have very little in common.

How did you establish that view? it seems to be an opinion and an imprecise one at that. It also now seems you don't doubt my capacity to design and build a real working compiler then, but you doubt my ability to "select concepts" to include in a new language, I can select whatever I want to select though, I started the thread.

From your statements in this thread, it appears that your experience is limited to a relatively narrow subset of the conceptual types of languages in existence[1].

Which of my "statements" are you interpreting that way? This sounds like a "No true Scotsman" argument in the making frankly.

You fail to engage with suggestions that would widen (possibly beneficially) your understanding. For example, you've completely ignored the possibility of your language including one facility that is very useful in resource-constrained embedded systems: fixed point arithmetic.

Would you like me to show you the posts where I advocated fixed point binary and decimal types and arithmetic? perhaps you overlooked them, if I showed them to you they'd prove you wrong on this specific point wouldn't they? Seriously, just say the word and I'll show you those posts.

I don't remember you discussing how multicore/multithread parallel processing will be expressed in your language; that will be critically important in the future.

If you want to discuss that then do so, have you asked me specifically about this area and been ignored? Why do you think I should be raising things that you regard as relevant, just raise them. Berating me for not raising points that you want to raise seems frankly pathetic.

I try to become less inexperienced in the things that matter to me. Knowing how and when to use languages that are fundamentally and conceptually very different is one such case. OTOH, implementing a compiler is completely uninteresting to me.

So just to be very clear you've never personally designed and written lexical analyzers, grammars, parsers, code generators or optimizers then?

[1] e.g. C, Pascal, PL/M, Delphi, Coral-66 are all procedural languages with little difference between them. Conceptually very different languages are VHDL, LISP, Smalltalk, Forth, OCCAM/xC etc etc.

I think you meant that the former are "imperative" languages, but no matter, different languages are different, I'm glad you brought that to my attention.


Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 17, 2023, 08:45:19 pm
Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?

I don't get the premise of your question nor the question itself. It looks like some users that have replied to this have taken it as "what are the features of C that make it appropriate as a system programming language", but that is not what I read in the above, that's not what you asked.

The exact answer to your exact question is obvious - C was explicitely designed to implement an OS. It doesn't need a "status", it's just what it was meant for. And history has proven a long track record of it being appropriate indeed for OS development and low-level stuff in general.

That doesn't mean it's "perfect", or that it's the only language for such endeavours. Other languages have been clearly used, as we already mentioned. A good chunk of the initial MacOS, and base applications was written in Pascal. Wirth designed Oberon as a language for implementing the Oberon OS. Ada has been extensively used for embedded development, and there are a few OSs written in Ada as well.

If the question was "what features of C are key features for a system prog language", then the answer is relatively simple. It has high-level features on par with languages such as Pascal (so not very high-level, bug high-level enough to express abstraction), but can get low-level enough (yes, in particular through the use of pointers, pointer arithmetic, logic operators, function pointers, ...) to address hardware almost directly.

Any modern language that wants to do the same usually resorts to "unsafe" blocks of code, such as Rust, but the idea of unsafe code was already there in Modula-3 - I suggested reading the language report which can be found online. The point is that once you have to use your language for system programming, you'll have to add ways of circumventing all your nice protections by adding "unsafe"constructs.

While that may look like a reasonable approach - instruct the developers to use safe code as much as possible, and restrict unsafe code to the bare minimum - the same can be achieved using multiple languages. Using a very high-level language along with assembly, or along with C for the very low-level code, also works, and is not significantly better or worse than trying to stuff all source code with a single language and resorting to tricks to achieve that.

Just my 2 cents.

We need a definition of "safe" and "unsafe" - do you have one? C# and Java have one but it's inapplicable here. In C# it means:

Quote
Most of the C# code you write is "verifiably safe code." Verifiably safe code means .NET tools can verify that the code is safe. In general, safe code doesn't directly access memory using pointers. It also doesn't allocate raw memory. It creates managed objects instead.

It's impossible to write safe code that can corrupt memory, I think that's what this boils down to. Corruption of memory is only possible (that's possible not inevitable) in unsafe code.

Now corrupt of course would need a definition too! But it includes altering memory not "owned" by the executing program or not freeing memory systematically, allowing heap exhaustion, memory leaks.

But what do you mean by "safe"?

By the way all I'm asking here is what core language capabilities would another language need to have, for it to be at least as "good" as C for systems programming, say writing an OS or running on an MCU manipulating A/D converters, UARTS etc with DMA and so on.

Note with the PL/I "model" I'm borrowing here, modifying a pointer's value through a raw arithmetic computation, is to be avoided, eliminated even. In C there are two reasons (so it seems to me) to compute a pointer:

1. As a means of accessing some datum relative to some other (e.g. accessing char array elements given the base address of the array and the element size)
2. Access data in some way not otherwise permitted by the langauge/types.

I might be wrong, but that's my thinking.

If we have no explicit pointer computation and do it all implicitly, why is that bad? why is it inevitable there'll be some cost? surely whatever we code in C can be better coded and at least as fast, by the code gen, the language

But of course take an STM32 device, that (in C) has huge headers with lots n lots of hard coded addresses, raw numbers, that is a need that a language must somehow cater for.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 17, 2023, 09:39:07 pm
In the coming weeks I expect to have a test parser for this language. I will produce a .Net Core console app that can be run easily in windows with the cmd-line. I'll also include a set of contrived source files that represent variations on valid inputs. Anyone will be able to create a source file and parse it.

If the parser reports a problem yet the code looks legal then we have a grammar bug.
If the parser doesn't report a problem yet the code looks illegal we have a grammar bug.

The goal of such tests is to break the parser, expose faults, hit it hard, hammer it with the most brutal source code that one can.

So this is just an FYI, to let anyone genuinely interested know ahead of time.

I very much value some of the inputs people have posted here, yes there's "noise" but also some extremely helpful points getting raised, I do appreciate it.



Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 17, 2023, 10:25:33 pm
Algol68 was the language that most influenced PL/I which I've spoken about at length here, there was also a build of it that accepted Russian keywords, I wonder if you were aware of that.
I was aware of the Russian Algol68 variant thanks. PL/1 never made an impression on this side of the pond. The nearest was a brief flurry of interest in PL/M in the late 70s.
Are you claiming there were no computer installations in the UK that used PL/I?

Read what I wrote.

Quote
You have seemed to be interested in expressing bit operations. Which of the 144 examples are you thinking of following. Do you prefer the VHDL or Smalltalk concepts? If none of the 144, what will be in your 145th variant?

I'm considering symbolic operators for left/right logical shift, right arithmetic shift, rotate left and rotate right, and possibly population count, is that what you're asking me?

That's part of it. Have a look at VHDL and Smalltalk, see what operations can be done on what, see how operations are expressed.

Quote
Quote
I have to ask, politely, would you please stop insinuating that I am "unqualified" in some way or other, to discuss this subject? Repeatedly making disparaging remarks and insulting comments is really not what I expect in any forum like this.

It isn't necessarily disparaging to regard someone as unqualified; we have all been unqualified. The key point is what any such person does or does not do to become more qualified. "Learning the literature/history" is necessary.

The key point is that you are conflating (no doubt intentionally) one's certifications with one's argument.

Er, no. This may be a divided by a common language issue.

There is little relationship between a formal certification and how well qualified someone is to make a judgement about something. We've all seen people with bits of paper that shouldn't be let near a keyboard or soldering iron. I'm not certified to be a father, but I am qualified to be one to my daughter.

Quote
Quote
I went to the trouble in my very first post to mention that I was experienced with compiler design and development, I anticipated that some might be skeptical, that a language is a large undertaking and not to be treated glibly, I know all that and therefore summarized what I had done before.

Yet you continue to insinuate I'm some kind of idiot.
Implementing a compiler and selecting which concepts to use/discard when creating a language have very little in common.
How did you establish that view? it seems to be an opinion and an imprecise one at that. It also now seems you don't doubt my capacity to design and build a real working compiler then...

I'm not surprised you don't understand there is a significant difference. That is consistent with your other statements.

I have never questioned your ability to implement a compiler.

Quote
... but you doubt my ability to "select concepts" to include in a new language, I can select whatever I want to select though, I started the thread.

Of course you can select concepts to include in your language. That's not at issue.

The issue is whether the concepts are interesting and/or novel - or just a recapitulation of the past without interesting advantages.

Quote
You fail to engage with suggestions that would widen (possibly beneficially) your understanding. For example, you've completely ignored the possibility of your language including one facility that is very useful in resource-constrained embedded systems: fixed point arithmetic.

Would you like me to show you the posts where I advocated fixed point binary and decimal types and arithmetic? perhaps you overlooked them, if I showed them to you they'd prove you wrong on this specific point wouldn't they? Seriously, just say the word and I'll show you those posts.

If you have, and I've no reason to doubt your word, then I've missed them - probably because they have been drowned out by posts about minor tweaks to things that are already commonplace.

Quote
I don't remember you discussing how multicore/multithread parallel processing will be expressed in your language; that will be critically important in the future.

If you want to discuss that then do so, have you asked me specifically about this area and been ignored? Why do you think I should be raising things that you regard as relevant, just raise them. Berating me for not raising points that you want to raise seems frankly pathetic.

Such things have been discussed. Nobody has been so crass as to specifically ask for your comments.

Quote
I try to become less inexperienced in the things that matter to me. Knowing how and when to use languages that are fundamentally and conceptually very different is one such case. OTOH, implementing a compiler is completely uninteresting to me.

So just to be very clear you've never personally designed and written lexical analyzers, grammars, parsers, code generators or optimizers then?

Certainly not. They are a solved topic within the constraint of a language's specification. They are a vanishingly small part of implementing interesting useful systems, not far off being lost in the noise. I have the same lack of interest in databases, for similar reasons.

I am, however, interested in
And especially how all those interact. Examples: the Mill architecture and DEC Alpha machines with C, and xC/xCORE with hard realtime systems.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 17, 2023, 11:05:09 pm
I was chatting to the Zig developers earlier, they have replaced the crude C preprocessor with something much better. They allow you to write code (in Zig) that can be executed at compile time and do stuff that's quite impressive. In essence this is very similar to PL/I's preprocessing language and in fact not really a different concept, but preprocessing code and Zig's "comptime" code is similar.

https://kristoff.it/blog/what-is-zig-comptime/

These are very interesting ideas...

Title: Re: The Imperium programming language - IPL
Post by: westfw on January 18, 2023, 12:21:27 am
I’ve always been nonplussed that the average assembler has a much more capable macro language than the C preprocessor.

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 18, 2023, 03:50:11 pm
Regarding keywords and reserved words. The draft Imperium grammar now has 66 keywords, these are basically tokens in the language that have lexical relevance for parsing rules.

But as I explained, it is trivial to add new ones, nothing can ever break, so that's how the list as been growing as I add keywords like "alias" or "coroutine" or "interrupt" keywords absent from conventional PL/I.

Now consider C, most source say it has (or originally had) 32:

Code: [Select]

auto double int struct
break else long switch
case enum register typedef
char extern return union
const float short unsigned
continue for signed void
default goto sizeof volatile
do if static while

all well and good, but they added 5 for C99 and then 7 for C11, these were real, justifiable, useful language enhancements, but look at how they HAD to be named (and there was always a real risk that some code somewhere had used such names for a type or variable too!).

Code: [Select]
_Alignas _Generic _Thread_local
_Alignof _Noreturn
_Atomic _Static_assert

If this had been Imperium they could have literally added "alignas" and "noreturn" (or "no return" if they'd wanted) and "thread local" (yes with a space even!) and all code, all prior code would compile fine, it is impossible to break a compile by adding a new keyword.

I recently tweaked the grammar to allow an optional keyword to follow and "end" like "end procedure" or "end loop" or "end if" that took two minutes and again, was fully backward compatible.

This is just another reminder of why it is essential to get the grammar right, solid, flexible and extensible BEFORE we do much else.



Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 18, 2023, 05:44:25 pm
I’ve always been nonplussed that the average assembler has a much more capable macro language than the C preprocessor.

Pre-processors and macros are of significant benefit when creating solutions in assembler.

They slightly simplified the creation of compilers in C, and were a useful hack in 64kbyte machines. At that stage C was regarded as a portable assembler. Nowadays, when C is used for large programs, pre-processors and macros become part of the problem rather than part of the solution.

We have learned a lot over the decades, including  languages which can structure inplementations of solutions. None of the moderbn languages which embody that hard-won experience have pre-processors, for very good reason.

(Clearly I exclude C++ from such modern languages!)
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 18, 2023, 06:20:50 pm
Visual Studio Code

I just discovered that IBM supply a VSCode extension that is aware of their PL/I language, it has syntax highlighting, code completion and more. Here's a quick test drive attached.

[attachimg=1]

This is excellent news because it proves that VSCode extension support is possible for languages that have no reserved words and that code completion is also possible for these languages. In that example the editor reports no syntax errors but has highlighted the variables "if", "then" and "else" as keywords, so that's either a bug in the extension or perhaps a setting alerting me to the fact these identifiers also match keywords, legal but usually not intentionally done.


Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 18, 2023, 08:35:01 pm
Multiway branches

The PL/I language has a more powerful construct than the C "switch" statement. I'm going to leverage that and include it in imperium, it's called a "select" statement, here's an example:

Code: [Select]

select (state);
when (1,4,7)
   call flush_buffers(-1);
when (2,3)
   call add_to_buffer(state, status);
   if status > 0 then
      call abort;
when (get_latest_failure(state))
   call log("There was at least one failure");
else
   call log("All is well");
end;


I've modified it a bit and used "else" rather than the PL/I "otherwise" it seems logical to do that since the meaning of "else" and "otherwise" is pretty much the same and the grammar has no issues using that keyword in two types of statements.

There are two important differences, to C's "switch"

1. There is no "fall through" the when clause ends and that's the end of the whole select block.
2. The expressions inside the "when" clauses do not need to be constants, any expression is fine, even a function call.
3. The "when" can also be followed by a "any" or an "all" with obvious meaning.

Ideas, suggestions about this feature are welcome.






   


Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 18, 2023, 10:09:52 pm
Operator symbols

Given that I want to support arithmetic/logical shift left/right along with rotate left and right, the question emerges as to what priorities these operators should be given as well as the actual symbol to use for them.

For rotation these sprang to mind:

Rotate left: <@<
Rotate right: >@>

The circularity of the @ seems a reasonable sign for rotation.

Anyway, suggestions welcome.

OK I settled this, these are the tokens for rotate:

Code: [Select]
L_ROTATE:     ('<<@');  // rotate: left bit rotated out rite bit becomes that rotated left bit
R_ROTATE:     ('@>>');  // rotate: rite bit rotated out left bit becomes that rotated rite bit

Here the @ symbol represents the right/left bit who's value is being rotated "in" so to speak.

Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 18, 2023, 10:44:26 pm
This is just so stupid.

The number of times rotate is used in programs, there is absolutely no disadvantage in typing rotate_right_through_carry(x, i). There is no point in inventing cryptic symbols and discussing what their parsing precedence should be.

Just total bike-shedding.

ISAs that aren't legacy from 8/16 bit machines from the 70s/80s often don't even HAVE a rotate instruction anyway (or a carry flag).
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 18, 2023, 11:02:12 pm
This is just so stupid.

The number of times rotate is used in programs, there is absolutely no disadvantage in typing rotate_right_through_carry(x, i). There is no point in inventing cryptic symbols and discussing what their parsing precedence should be.

Just total bike-shedding.

ISAs that aren't legacy from 8/16 bit machines from the 70s/80s often don't even HAVE a rotate instruction anyway (or a carry flag).

Ahh, yet another offensive, rude, narrow minded post...

Your remarks are those of the knuckle dragger.

How many times is a rotate operation used in programs? seriously? If no language offers a "rotate" operator then HOW THE **** COULD THERE BE any examples of it being used!!

Quote from: ARM
The ARM processor has two rotate operations, ROR (Rotate Right) and RRX (Rotate Right with Extend).

and

Quote
The ARM core contains a Barrel shifter which takes a value to be shifted or rotated, an amount to shift or rotate by and the type of shift or rotate. This can be used by various classes of ARM instructions to perform comparatively complex operations in a single instruction. On ARMs up to and including the ARM6 family, instructions take no longer to execute by making use of the barrel shifter, unless the amount to be shifted is specified by a register, in which case the instruction will take an extra cycle to complete.

Take a look at the ARM reference manual, you might learn something, clearly you've just learned something about ARM's ISA that you rather obviously didn't know.

Here's a tip Mr. Halt, if you disagree then say so, don't insult and disparage, just be diplomatic, why must you write "this is just so stupid" rather than "well you might want to consider that..." or "that's certainly interesting, but what if the use cases are very few and far between?" either of which responses would have been better.

The language is new, therefore the notation for a host of operations is an open discussion point, even if some ISAs do not support a hardware rotate instruction so what? the language can still support it. Many of the MCUs out there doing floating point arithmetic do not have instructions for floating point arithmetic so should we not include floating point in a language?

Finally, it is far better to use a dedicated symbol than a culture specific word where possible (clearly you've zero experience of language design). We use + rather than "plus( )" for that reason, have you really never noticed this?

Would you advocate something more like:

Code: [Select]

counter equals zero;

if counter greater than ten then
   counter equals twenty;

total equals counter plus prior_count;

would that be more to your liking? perhaps you'd care to port COBOL to the world of MCUs?

Title: Re: The Imperium programming language - IPL
Post by: cfbsoftware on January 18, 2023, 11:08:19 pm
Multiway branches
Ideas, suggestions about this feature are welcome.
Consider also including ranges as in the Oberon CASE statement e.g. using your syntax:

when ("a".."z","0"..9")
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 18, 2023, 11:11:52 pm
Multiway branches
Ideas, suggestions about this feature are welcome.
Consider also including ranges as in the Oberon CASE statement e.g. using your syntax:

when ("a".."z","0"..9")

Thanks, this is indeed a good idea, in fact range notation is now very common but was unheard of in the 60s when PL/I was nurtured, I will definitely look at this soon!

Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 18, 2023, 11:28:27 pm
How many times is rotate used in programs? seriously? If no language offers a "rotate" operator then HOW THE **** COULD THERE BE any examples of it being used!!

Very very very rarely.

Rotate is used if and only if the algorithm calls for it, nothing at all to do with the programming language or instruction set.

Quote
Take a look at the ARM reference manual, you might learn something, clearly you've just learned something about ARM's ISA that you rather obviously didn't know.

I've been programming ARM CPUs in assembly language since the 1990s, maybe late 80s.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 18, 2023, 11:30:12 pm
This is just so stupid.

The number of times rotate is used in programs, there is absolutely no disadvantage in typing rotate_right_through_carry(x, i). There is no point in inventing cryptic symbols and discussing what their parsing precedence should be.

Just total bike-shedding.

ISAs that aren't legacy from 8/16 bit machines from the 70s/80s often don't even HAVE a rotate instruction anyway (or a carry flag).

Knowing why that is the case would require that the OP understood the evolution in processors that occurred then.

The OP is having fun creating a "VAX language" when other people are creating a "RISC language" :)  Fettling with a language is a harmless hobby.

Rotate is useful for a few DSP/CRC class algorithms. But those would benefit more from saturating arithmetic or fixed point arithmetic.
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 18, 2023, 11:34:26 pm
Fettling with a language is a harmless hobby.

Oh yeah, it's fun and harmless. Well, harmless until it mysteriously escapes the computer of its author and starts invading universities and private companies, sometimes for an unknown reason. :-DD
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 18, 2023, 11:36:03 pm
Knowing why that is the case would require that the OP understood the evolution that occurred then.

I'm curious what combination of characters he is planning to use for:

- CRC32 instruction

- DES and AES instructions

- SHA

- bit reversal in a word

- byte reversal

- selecting bits from one or another source based on a mask

I could go on. There is an infinity of operations, and even hundreds of operations that have been implemented as instructions in some ISA or another. Each very handy and performance-enhancing when needed -- which might be exactly once on the whole machine, in one function in the (hand-written) standard library.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 18, 2023, 11:37:07 pm
How many times is rotate used in programs? seriously? If no language offers a "rotate" operator then HOW THE **** COULD THERE BE any examples of it being used!!

Very very very rarely.

Rotate is used if and only if the algorithm calls for it, nothing at all to do with the programming language or instruction set.

Mr. Halt, addition is only used if the algorithm calls for it, arithmetic shift is only used if the algorithm calls for it.

Quote
Take a look at the ARM reference manual, you might learn something, clearly you've just learned something about ARM's ISA that you rather obviously didn't know.

I've been programming ARM CPUs in assembly language since the 1990s, maybe late 80s.

Yet you never realized they supported a rotate operation? hmm...





Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 18, 2023, 11:43:08 pm
Knowing why that is the case would require that the OP understood the evolution that occurred then.

I'm curious what combination of characters he is planning to use for:

- CRC32 instruction

- DES and AES instructions

- SHA

- bit reversal in a word

- byte reversal

- selecting bits from one or another source based on a mask

I could go on. There is an infinity of operations, and even hundreds of operations that have been implemented as instructions in some ISA or another. Each very handy and performance-enhancing when needed -- which might be exactly once on the whole machine, in one function in the (hand-written) standard library.

Rotates are a very small step away from shifts, very very small difference so if we have symbols for left and right shifts its very reasonable to have them for rotates too (which is why the symbols ARE SHIFTS with an additional character), in mathematics and physics we value symmetry, a shift is just a rotate with a discard implied, think about these things before you wade into such discussions like a bull in a china shop, not everyone sees the world through your blinkered, jaded spectacles.

(You even sarcastically mention byte reversal, completely failing to grasp the need for big/little endian capabilities...)

What about XOR? what symbols (if any!) would one use for that in a new language striving to be more helpful than C on hardware programming?
Title: Re: The Imperium programming language - IPL
Post by: ataradov on January 18, 2023, 11:58:18 pm
Rotates are a very small step away from shifts, very very small difference
They are not though. Shifts have the same meaning for various word lengths. Rotates are dependent on the type of the underlying value, so the result depends on how abstract machine and types are implemented.  And that also poorly maps on most hardware.

And what you are doing is really bike-shedding. None of what you are doing is important in a language design.

What about XOR? what symbols (if any!) would one use for that in a new language striving to be more helpful than C on hardware programming?
C has XOR.

And you calling people to look at things is just funny. It is good that you are discovering all those things and all, but it just looks really funny from outside.

Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 12:07:44 am
And you calling people to look at things is just funny. It is good that you are discovering all those things and all, but it just looks really funny from outside.

yup, precisely.

It looked troll-ish, but - reconsidering the whole picture, it's just someone who really believes "EXE" is the binary format used in MPU toolchain

LOL :-DD

I wanted to be polite, and reported the only weird case where "EXE" is actually like that.
(SONY's choice, it's always funny to note)

Well, but my blood boiled when I saw labels and gotos, exposed without the minimal concept of RTE.
I can acdept goto in the linux Kernel because Linus knows what he is doing, otherwise, that's really no-goooooooooooooood and harmful.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 12:12:11 am
Rotates are a very small step away from shifts, very very small difference
They are not though. Shifts have the same meaning for various word lengths. Rotates are dependent on the type of the underlying value, so the result depends on how abstract machine and types are implemented.  And that also poorly maps on most hardware.

And what you are doing is really bike-shedding. None of what you are doing is important in a language design.

What about XOR? what symbols (if any!) would one use for that in a new language striving to be more helpful than C on hardware programming?
C has XOR.

And you calling people to look at things is just funny. It is good that you are discovering all those things and all, but it just looks really funny from outside.

Not as I'm currently defining these operations. I define rotate exactly as it is defined here:

Quote from: ARM
Rotate right (immediate) provides the value of the contents of a register rotated by a variable number of bits. The bits that are rotated off the right end are inserted into the vacated bit positions on the left.

Right C uses the symbol ^ for exclusive or, but Imperium uses that that for arithmetic exponentiation. So try again, what symbol could you suggest for exclusive OR if ^ is not available?

As for "funny" what's funny is that Mr. Halt claimed almost no processors support rotate yet I showed him that ARM actually does, and you claiming that rotate is not very close to shift until I - just now - showed you that it is.




Title: Re: The Imperium programming language - IPL
Post by: ataradov on January 19, 2023, 12:14:07 am
Not as I'm currently defining these operations. I define rotate exactly as it is defined here:
That ARM operation is only defined for 32-bit words. What if I want to rotate a byte?

As for "funny" what's funny is that Mr. Halt claimed almost no processors support rotate yet I showed him that ARM actually does, and you claiming that rotate is not very close to shift until I - just now - showed you that it is.
LOL
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 12:14:53 am
Rotates are a very small step away from shifts, very very small difference
They are not though. Shifts have the same meaning for various word lengths. Rotates are dependent on the type of the underlying value, so the result depends on how abstract machine and types are implemented.  And that also poorly maps on most hardware.

And what you are doing is really bike-shedding. None of what you are doing is important in a language design.

What about XOR? what symbols (if any!) would one use for that in a new language striving to be more helpful than C on hardware programming?
C has XOR.

And you calling people to look at things is just funny. It is good that you are discovering all those things and all, but it just looks really funny from outside.

You're being ignored.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 12:15:28 am
Knowing why that is the case would require that the OP understood the evolution that occurred then.

I'm curious what combination of characters he is planning to use for:

- CRC32 instruction

- DES and AES instructions

- SHA

- bit reversal in a word

- byte reversal

- selecting bits from one or another source based on a mask

I could go on. There is an infinity of operations, and even hundreds of operations that have been implemented as instructions in some ISA or another. Each very handy and performance-enhancing when needed -- which might be exactly once on the whole machine, in one function in the (hand-written) standard library.

You're being ignored.

Title: Re: The Imperium programming language - IPL
Post by: ataradov on January 19, 2023, 12:20:22 am
Sure ignore all you want. BTW, modern ISA (RISC-V) don't have rotate. It is not necessary and poorly maps on the high level code due to that register size vs variable size issue.

And that's why if you look at the intrinsic functions provided by the compilers, you will see a mess like __rolb/__rorb for 8-bit, __rolw/__rorw (16-bit), __rold/__rord (32-bit), __rolq/__rorq (64-bit). There is no one universal rotate. And if you do the "manual" rotate from the code, optimizing compiler will recognize the pattern and automatically generate appropriate rotate code, which in many cases will not involve hardware rotate instruction.

In case of RISC-V recently ratified B extension does add rotates, but it is just a minor part of a massive set of actually useful bit manipulation extensions.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 19, 2023, 12:22:25 am
And what you are doing is really bike-shedding. None of what you are doing is important in a language design.

Nor to someone using a language to drain a swamp.

But it is important to someone principally interested in exploring the internal components in a compiler.

Quote
And you calling people to look at things is just funny. It is good that you are discovering all those things and all, but it just looks really funny from outside.

It does. And sad.

It is always sad when someone doesn't understand what is driving them to a particular course of action.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 19, 2023, 12:29:44 am
You're being ignored.

Not quite. The multiple people you have put in that category are not being ignored. They are being being listened to by all but one person.

If there is no body that you are listening to, why are you continuing to post in this thread?
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 12:39:17 am
It's fine, absolutely fine to disagree with someone. That someone might be right, they might be wrong, or it might be a matter of opinion, hard to decide.

What's not fine is rudeness, accusations of stupidity and other insults and hurtful remarks, which some here seem to perversely enjoy, seem to know no other way to disagree.

If you spoke to someone in person, in a professional meeting with others present or as a visitor to a company, the way you express yourselves here then you'd risk being fired. Anyone who relies on insults and belittling language to communicate with people they disagree with is likely a huge liability in a management role, an HR accident waiting to happen.

But it matters not to me, if you disagree and expect me to respect your opinion, your expertise, to listen to and consider your views then that will never happen when you begin by insulting me, when that happens I stop listening, so any value you might have had is gone, lost.

If that's your objective then you've achieved it.





Title: Re: The Imperium programming language - IPL
Post by: westfw on January 19, 2023, 12:51:49 am
Quote
ISAs that aren't legacy from 8/16 bit machines from the 70s/80s often don't even HAVE a rotate instruction anyway (or a carry flag).
That seems like a bit of an exaggeration.  MIPS, Alpha, and RISC-V lack carry flags, but ARM, Sparc, and PPC all have one (but sparc doesn't have Rotate.)

(Carry seems like a lot to give up.  If the CC register is a potential bottleneck, how come you don't see it implemented by adding an extra bit to each register?)
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 19, 2023, 12:54:05 am
And what you are doing is really bike-shedding. None of what you are doing is important in a language design.

Triviality of the issue being discussed is one aspect of it. The other is not being competent to understand and discuss the more important issues.

"Parkinson provides the example of a fictional committee whose job was to approve the plans for a nuclear power plant spending the majority of its time on discussions about relatively minor but easy-to-grasp issues, such as what materials to use for the staff bicycle shed"
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 19, 2023, 01:01:48 am
Quote
ISAs that aren't legacy from 8/16 bit machines from the 70s/80s often don't even HAVE a rotate instruction anyway (or a carry flag).
That seems like a bit of an exaggeration.  MIPS, Alpha, and RISC-V lack carry flags, but ARM, Sparc, and PPC all have one (but sparc doesn't have Rotate.)

I said "often" not "always".

Quote
(Carry seems like a lot to give up.  If the CC register is a potential bottleneck, how come you don't see it implemented by adding an extra bit to each register?)

Using 63 bits per limb in your bignum code instead of 64 gives the same effect.
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 19, 2023, 01:10:37 am
Adding an extra bit per register as carry is something I suggested a while ago in my thread about RISC-V.

The whole question, as always, is whether it's worth adding that in general use, rather than making it an ISA extension.

Unless you're dealing with "big integers", arithmetic operations with carry are not that useful, especially on a 64-bit CPU. (But even on a 32-bit CPU which is a MCU, that's not often that useful.) So that's kind of a niche (albeit a nice to have if you are dealing with that.)

As to bitwise rotate, it's already in an extenson for RISC-V (bitmanip). It doesn't require a "carry" bit, as the carry does not stick after the operation is completed.
Title: Re: The Imperium programming language - IPL
Post by: ataradov on January 19, 2023, 01:20:55 am
It was probably debated to death elsewhere, but it is important to differentiate instruction set from the microarchitecure. Typical RISC-V add/sltu pairs may be more code than ARM's add/adc. but they don't have to take more time to execute if performance matters. Those instructions may be fused and executed together. But symmetrical register-only architecture is potentially easier to implement in heavily pipelined and OOO designs. It is hard to tell which one is really better.

X86 is pretty bad ISA, yet until recently its implementations absolutely destroyed everything else. Then Apple managed to optimize ARM quite well. I'm not sure if someone would manage to get anything close from RISC-V. We'll have to wait and see.
Title: Re: The Imperium programming language - IPL
Post by: westfw on January 19, 2023, 01:25:13 am
Quote
Unless you're dealing with "big integers", arithmetic operations with carry are not that useful
I guess.  But aren't "big integer" operations become more common, what with assorted crypto stuff?
Quote
Using 63 bits per limb in your bignum code instead of 64 gives the same effect.
I guess.  I was imagining a small number of instructions that would access the 65th bit, so that the normal instruction set could be used while being blissfully unaware that  the 65th bit was there.
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 19, 2023, 01:31:22 am
It was probably debated to death elsewhere, but it is important to differentiate instruction set from the microarchitecure. Typical RISC-V add/sltu pairs may be more code than ARM's add/adc. but they don't have to take more time to execute if performance matters. Those instructions may be fused and executed together. But symmetrical register-only architecture is potentially easier to implement in heavily pipelined and OOO designs. It is hard to tell which one is really better.

Yep. And yes this has been debated a lot. Instruction fusing may be costly to implement depending on your architecture indeed. I would personally go for the extra bit per register option myself as an extension for my own core. But since its main benefit would be for pretty specific stuff (such as big ints/"arbitrary precision" computation), designing a specific "accelerator" may be a better option anyway rather than trying to stuff more instructions and get only marginal performance increase in the end.

X86 is pretty bad ISA, yet until recently its implementations absolutely destroyed everything else. Then Apple managed to optimize ARM quite well. I'm not sure if someone would manage to get anything close from RISC-V. We'll have to wait and see.

x86 is pretty bad as an instruction set, but modern Intel CPUs are very far away from the direct map to microachitecture that was the case for early x86 CPUs. Nowadays they are RISC machines with complex translation units.

What Apple did with the M1/M2 is remarkable - the missing market for ARM was precisely that of the desktop, and it's now wide open. Now the top of the line Intel CPUs still crush the Apple CPUs by a large margin, but that's already a great achievement.

Title: Re: The Imperium programming language - IPL
Post by: ataradov on January 19, 2023, 01:47:14 am
I would personally go for the extra bit per register option myself as an extension for my own core.
This only works as long as you are not using it in any other context. As soon as you want to branch on that carry bit, you are screwed again. And the add/adc use seems pretty limited to bother.

I just downloaded Armv9-A ARM and holy crap, it is 12000 pages long. Good luck implementing that.
Title: Re: The Imperium programming language - IPL
Post by: Nominal Animal on January 19, 2023, 03:05:55 am
Carry is much less useful than one would initially think.  I used to think it important, until discussing the exact details (of architectures without flags) with brucehoult, and looking carefully at the machine code generated on different ISAs for my typical use cases and patterns.  It turns out I just want ABIs that can pass more than one scalar value in registers, like SysV on x86-64.

For example, when converting IEEE 754 floating-point numbers to octal/decimal/hexadecimal exactly correctly, it turns out that using 28-bit limbs in 32-bit registers (or 60-bit limbs in 64-bit registers on 64-bit architectures) is much more efficient than full 32-bit limbs, because most operations involve just one digit (in base 8/10/16) at a time.  The difference is definitely significant, so much so that I fully believe the 14.3% extra memory use is worth it.

This is exactly why I believe it is so important to examine what kind of machine code one wants a compiler to produce.
Basic assumptions (like "condition codes are necessary for efficient code doing X") tend not to hold under close scrutiny.
Title: Re: The Imperium programming language - IPL
Post by: westfw on January 19, 2023, 07:27:18 am
The last thing I really liked having Carry and ADC for was an efficient implementation of the IP checksum.
It's probably pretty academic if you have 64bit math, but it was nice on 8 and 16bit CPUs. (half the size and twice the speed for an AVR. (https://github.com/WestfW/Duino-hacks/blob/master/ipchecksum_test/ipchecksum_test.ino))
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 11:06:49 am
If there is no body that you are listening to, why are you continuing to post in this thread?

must be the new fashion web hit: write-only mode  :o :o :o
(null pipein, flow pipeout, dunno pipeerr
like Facebook, I mean)
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 11:14:50 am
The last thing I really liked having Carry and ADC for was an efficient implementation of the IP checksum.

I did the same on a 68hc11 board with CS8900 10Mbits
Add with Carry to A ADCA
Add with Carry to B ADCB

great trick, found an old article on a UK magazine about that ;D
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 12:10:00 pm
This is exactly why I believe it is so important to examine what kind of machine code one wants a compiler to produce.

Yup, assolutely.

I mean, at least one should know that "grammar" is only a part of the puzzle. Then you have to deal with the "machine layer", and matching them on purpose (for which purpose?) is not as easy as one thinks.
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 12:13:22 pm
Typical RISC-V add/sltu pairs may be more code than ARM's add/adc

I have { add, sltu } pairs on my MIPS-5++ Atlas board ;D
Sounds good choice, good pairs!
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 19, 2023, 12:22:03 pm
I asked ChatGPT "Let's assume that you need to design a new programming language for embedded systems that will be targeted for safety-critical and mission-critical embedded systems, and provide safe memory handling and safe array operators. What would it be like?"
Quote
A programming language for embedded systems that is targeted for safety-critical and mission-critical systems would likely have the following features:

1. Memory safety: The language would have built-in mechanisms to prevent common memory-related errors such as buffer overflows, use-after-free, and null pointer dereferences.

2. Type safety: The language would have a strong type system that helps prevent type-related errors, such as type casting errors and implicit type conversions.

3. Array bounds checking: The language would have built-in array bounds checking to prevent out-of-bounds errors when accessing arrays.

4. Automatic memory management: The language would have a garbage collector or other mechanism for automatic memory management, which would help prevent memory leaks and other related errors.

5. Strict syntax and semantics: The language would have a strict syntax and semantics, in order to minimize the chances of introducing new errors.

6. Formal verification: The language would have formal verification tools that could be used to prove the absence of certain types of errors, such as data races.

7. Strong error handling: The language would have strong error handling, in order to prevent errors from propagating and causing further issues.

8. Support for concurrency: The language would have built-in support for concurrency, in order to make it easier to write multi-threaded code that is also safe.

9. Support for functional programming: The language would support functional programming features such as immutable variables, closures, and higher-order functions, which would help make the code more predictable and less error-prone.

10. Support for hardware-level control: The language would have support for interacting with low-level hardware, in order to allow for the precise control that is often required in embedded systems.

11. Support for real-time constraints: The language would have built-in support for real-time constraints, such as hard real-time and soft real-time, allowing developers to specify time constraints and deadlines for specific operations.

12. Interoperability: The language would have the ability to interoperate with other languages, such as C and C++, which are commonly used in embedded systems. This would allow for the use of existing libraries and codebases, and make it easier to integrate the new language with existing systems.

13. Standard libraries: The language would have a comprehensive set of standard libraries, including data structures and algorithms, for use in embedded systems.

14. Compact and efficient: The language would be designed to be compact and efficient, in order to minimize memory and processing requirements, which are important considerations in embedded systems.

15. Security features: The language would have built-in security features such as secure data handling and encryption to protect against common security threats.

16. Tooling and debugging: The language would have a robust set of tooling and debugging options, allowing developers to easily locate and fix errors.

17. Documentations: The language would have comprehensive and up-to-date documentation, including tutorials, guides, and reference material, to help developers get started quickly and become proficient with the language.

18. Community support: The language would have a strong and active community of developers, providing support and contributing to the language's development and maintenance.

Overall, the language would be designed with safety and reliability in mind, with a focus on preventing errors and ensuring that the code is predictable and easy to understand. It would also be efficient and flexible enough to meet the specific requirements of embedded systems.

19. Certification: The language would be designed to meet industry standards and certifications for safety-critical systems, such as ISO 26262 for automotive systems and DO-178C for avionics systems.

20. Safety-critical libraries: The language would have a set of libraries specifically designed for safety-critical systems that provide functionality such as fault-tolerance and redundancy.

21. Safety-critical constructs: The language would have constructs such as assertions and monitors that can be used to express safety properties and to check them at runtime.

22. Model-based development: The language would support model-based development, which allows for the design, verification, and validation of safety-critical systems using formal methods.

23. Traceability: The language would have traceability features that allow developers to trace requirements, design, and implementation, which is critical for safety-critical systems.

24. Auditing and logging: The language would have built-in support for auditing and logging which allows to check and trace the actions of the system and detect malfunctions that could cause safety hazards.

25. Testability: The language would be designed with testability in mind, allowing developers to easily create and execute unit tests, integration tests, and system tests to verify the correct functioning of the system.

Overall, the new programming language for embedded systems would be designed to meet the specific needs of safety-critical and mission-critical systems, providing a combination of safety, reliability, and efficiency, while at the same time meeting industry standards and certifications.

26. Design by Contract: The language would support Design by Contract (DbC) paradigm, which allows developers to express preconditions, postconditions, and class invariants that specify the expected behavior of the functions and classes, making it easier to understand and reason about the code.

27. Static analysis: The language would have built-in static analysis tools that can analyze the code and detect potential errors, such as null pointer dereferences, race conditions, and buffer overflows, before the code is even run.

28. Code review: The language would have tools and processes in place to support formal code reviews, where code is reviewed by other developers, in order to catch errors and improve the quality of the code.

29. Built-in safety mechanisms: The language would have built-in safety mechanisms such as watchdog timers, which can automatically reset the system if it becomes unresponsive, and fail-safe modes, which can be activated if a critical error is detected.

30. Support for formal verification: The language would support formal verification using tools such as model checkers, theorem provers, and static analyzers, which can be used to prove that the code meets certain safety properties.

31. Support for safety standards: The language would be designed to support safety standards such as IEC 61508 and ISO 26262, which provide guidelines for the development of safety-critical systems.

Overall, the new programming language for embedded systems would provide a comprehensive set of features for safety-critical and mission-critical systems, including memory safety, type safety, array bounds checking, automatic memory management, strict syntax and semantics, formal verification, strong error handling, support for concurrency, functional programming, and hardware-level control. Additionally, it would have features to support real-time constraints, interoperability, standard libraries, compact and efficient, security, tooling and debugging, documentation, community support and safety-critical libraries, constructs, model-based development, traceability, auditing and logging, testability, Design by Contract, static analysis, code review, built-in safety mechanisms, support for formal verification and safety standards.


This is a good list to keep in mind when designing a new programming language. Take existing programming languages used in safety-critical embedded systems (such as C, C++, Ada and Rust), implement their good features, and leave out and fix their problematic features. Also, make sure that the new programming language will meet requirements 5 and 18.
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 12:35:55 pm
I was imagining a small number of instructions that would access the 65th bit, so that the normal instruction set could be used while being blissfully unaware that  the 65th bit was there.

On my MIPS32 router, I am playing with a Crypto processor made by Soekris; it does all the math, block-in, does crypt/uncrypt, block-out. All DMA driven with a large buffer. A piece of cake, and it off-loads the CPU.

I think all the BIG-numbers and all the crypto stuff (VPN?) should be done this way!

I mean, by a dedicated Co-Processor, so you won't even care if { ADC, ROL, ROR, &C ... } is supprted in your CPU-ISA :-//
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 12:40:23 pm
I asked ChatGPT

Better get a job in avionics.
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 19, 2023, 01:01:09 pm
I asked ChatGPT

Better get a job in avionics.

Really, you are thinking that good design practices are beneficial only in avionics? If you have done any commercial software design and implementation, you should know by now that debugging and fixing programming errors is very time consuming and expensive. Preventing the common programming errors in the first place will be much cheaper. Therefore, the programming language should be designed such that it helps preventing making typical programming errors and blunders.

Maintaining existing code base and keeping it in good shape over multiple changes will also benefit from any help the programming language has to offer.
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 01:18:07 pm
Really, you are thinking that good design practices are beneficial only in avionics?

No, I think real life problems need real life training, while putting a list of things taken from Wikipedia (never seen in action) is... frankly not even close to learning anything useful.
Title: Re: The Imperium programming language - IPL
Post by: magic on January 19, 2023, 01:19:39 pm
I asked ChatGPT "Let's assume that you need to design a new programming language for embedded systems that will be targeted for safety-critical and mission-critical embedded systems, and provide safe memory handling and safe array operators. What would it be like?"
I don't know if AI will replace programmers anytime soon, but it will be a godsend for producing marketing materials :D
Title: Re: The Imperium programming language - IPL
Post by: JPortici on January 19, 2023, 01:21:08 pm
Also, make sure that the new programming language will meet requirements 5 and 18.

 :D

Quote
18. Community support: The language would have a strong and active community of developers, providing support and contributing to the language's development and maintenance.

impossible, everyone in the community will soon be in the ignore list

===============================================

By the way, Even though there is no rotate operator in C, compilers that target actitectures that have rotate instructions usually recognize particular constructs and translate them into rotate instructions

Quote from: XC8 Compiler Manual
Rotate operations can be performed in your code, despite the C language not including a rotate operator.
The compiler will detect expressions that implement rotate operations using shift and logical operators and compile
them efficiently.
For the following code:
c = (c << 1) | (c >> 7);
if c is unsigned and non-volatile, the compiler will detect that the intended operation is a rotate left of 1 bit and will
encode the output using the PIC MCU rotate instructions. A rotate left of 2 bits would be implemented with code like:
c = (c << 2) | (c >> 6);
This code optimization will also work for integral types larger than a char. If the optimization cannot be applied, or
this code is ported to another compiler, the rotate will be implemented, but typically with shifts and a bitwise OR
operation.

or provide a builtin that map to the dedicated instruction (but you usually want to use the rotate in algorythms you are already writing in assembly sooooo.....)
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 19, 2023, 01:29:21 pm
Also, make sure that the new programming language will meet requirements 5 and 18.

 :D

Quote
18. Community support: The language would have a strong and active community of developers, providing support and contributing to the language's development and maintenance.

impossible, everyone in the community will soon be in the ignore list

 >:D

I read that item 18 as follows: The new programming language needs to be able to attract enough critical mass so that it will be known & accepted, stay maintained and developed (like the popular programming languages have evolved over the years), get support from the existing and new tools, and will survive many years to come without becoming extinct.
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 19, 2023, 01:44:38 pm
I asked ChatGPT "Let's assume that you need to design a new programming language for embedded systems that will be targeted for safety-critical and mission-critical embedded systems, and provide safe memory handling and safe array operators. What would it be like?"
I don't know if AI will replace programmers anytime soon, but it will be a godsend for producing marketing materials :D

I think ChatGPT gave pretty good answer, considering the fact it took only few minutes to get that list. I would consider the current state of AI as an assisting technology, and the user needs to be able to evaluate the answers for sanity, correctness and usefulness.
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 01:52:42 pm
my HC12 book reports things in a very nice way

Logical Shift
Shift Left (Memory,A,B,D), {LSL,LSLA,LSLB,LSLD }
Shift Right (Memory,A,B,D), { LSR,LSRA,LSRB,LSRD }

Arithmetic Shift
(Similar to a Logical shift, but the sign bit remains unchanged)
Shift Left (Memory,A,B,D), { ASL,ASLA,ASLB,ASLD }
Shift Right (Memory,A,B,D), { ASR,ASRA,ASRB }

Cyclic Shift
Left (Memory,A,B), { ROL, ROLA,ROLB }
Right (Memory,A,B), { ROR, RORA,RORB }

Just, it calls "Cyclic Shift" what we call "bit Rotation", but it's the same.

I'd like to see what ICC12 (C compiler for Motorola 68HC12) would produce on a C piece of blow-fish algorithm :D
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 19, 2023, 01:58:27 pm
Really, you are thinking that good design practices are beneficial only in avionics?

No, I think real life problems need real life training, while putting a list of things taken from Wikipedia (never seen in action) is... frankly not even close to learning anything useful.

If you think about C and C++ languages for example, there are many recommendations and guidelines for producing maintainable and less buggy software (I am not talking about naming conventions and formatting here). For example MISRA-C and C++ contain a lot of recommendations that are result of a poor language design in the first place.

MISRA C/C++ recommendations are not publicly available, but similar kind of set of recommendations can be found online from ESCR: Embedded System development Coding Reference guide for C and C++languages, respectively:
https://www.ipa.go.jp/files/000065271.pdf (https://www.ipa.go.jp/files/000065271.pdf)
https://www.ipa.go.jp/files/000055042.pdf (https://www.ipa.go.jp/files/000055042.pdf)
Title: Re: The Imperium programming language - IPL
Post by: magic on January 19, 2023, 01:58:46 pm
I think ChatGPT gave pretty good answer, considering the fact it took only few minutes to get that list. I would consider the current state of AI as an assisting technology, and the user needs to be able to evaluate the answers for sanity, correctness and usefulness.
It gave you the answers you wanted to hear, which is its whole purpose ;)

The holy grail of marketing is here. But ask it to actually design something new and then :scared:
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 19, 2023, 02:05:06 pm
I think ChatGPT gave pretty good answer, considering the fact it took only few minutes to get that list. I would consider the current state of AI as an assisting technology, and the user needs to be able to evaluate the answers for sanity, correctness and usefulness.
It gave you the answers you wanted to hear, which is its whole purpose ;)

The holy grail of marketing is here. But ask it to actually design something new and then :scared:

And, what would be your answer to that same question then? According to your reply, you have some other properties that are more important in a programming language.
Title: Re: The Imperium programming language - IPL
Post by: Siwastaja on January 19, 2023, 02:13:21 pm
Quote
18. Community support: The language would have a strong and active community of developers, providing support and contributing to the language's development and maintenance.

impossible, everyone in the community will soon be in the ignore list

This, was quite hilarious seeing the OP rage-ignoring the few world class experts who happen to hang on this forum ;D. (You know who you are, of course.)

I can totally see in my mind where OP's "experience" is coming from: working as a middle manager who simply fires everybody who says "no" to his absolutely stupid ideas, and collects a bunch of so called "yes men" ( https://www.urbandictionary.com/define.php?term=yes%20men (https://www.urbandictionary.com/define.php?term=yes%20men) ) around him. And there is no shortage of yes men.

This is the classic pattern of destroying the productivity of a business, ignoring those who do understand, and spending all the time discussing the color and construction materials of the bike shed and micromanaging this discussion. And the business can appear to still run, based on buying out smaller companies with their product lines which are profitable for some time before getting destroyed. And/or outsourcing projects that need to be done, to smaller companies with different mindset.
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 02:13:52 pm
Nice to see the MIPS.Inc SDE assembly on my copy of MIPS Architecture For Programmers

"Rotate" is a real instruction that was introduced in the MIPS32R2 revision(1) of the MIPS Architecture
Code: [Select]
//Rotate Word Right                (SmartMIPS Crypto engine)
//The contents of the low-order 32-bit word of GPR rt are rotated right
//the word result is placed in GPR rd.
//The bit-rotate amount is specified by sa

        ROTR rd, rt, sa
but if you tell your assemble that your CPU is MIPS3 (.cpu MIPS3) then assembler compiler will accept it as "pseudoinstruction" that will replace with a sequence.

LOL, funny low level  ;D

(1) 2002/Release2
added+={ DI, EHB, EI, EXT, INS, JALR.HB, JR.HB, MFHC1, MFHC2, MTHC1, MTHC2, RDHWR, RDPGPR, ROTR, ROTRV, SEB, SEH, SYNCI, WRPGPR, WSBH }

ROTR Rotate Word Right (Release 2 Only)
ROTRV Rotate Word Right Variable (Release 2 Only)
SLL Shift Word Left Logical
SLLV Shift Word Left Logical Variable
SRA Shift Word Right Arithmetic
SRAV Shift Word Right Arithmetic Variable
SRL Shift Word Right Logical
SRLV Shift Word Right Logical Variable
Title: Re: The Imperium programming language - IPL
Post by: Siwastaja on January 19, 2023, 02:20:04 pm
I think ChatGPT gave pretty good answer, considering the fact it took only few minutes to get that list. I would consider the current state of AI as an assisting technology, and the user needs to be able to evaluate the answers for sanity, correctness and usefulness.
It gave you the answers you wanted to hear, which is its whole purpose ;)

The holy grail of marketing is here. But ask it to actually design something new and then :scared:

And, what would be your answer to that same question then? According to your reply, you have some other properties that are more important in a programming language.

It's a good list. A language model AI like that is pretty good in combining and summarizing widely accepted and widely known ideas, so basically a much more effective version of Google. It is also mostly correct, as it finds some kind of consensus. But all that information is something we already know (we, as in people who would be tasked with such safety critical embedded project, or designing a new language for such purpose). Possibly very helpful for someone who is trying to learn the basics of the subject, for example to understand what their team is doing, or building a team. If a designer completely ignores some of the points, or does the opposite, then it warrants a question: "why?"
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 02:35:48 pm
If you think about C and C++ languages for example, there are many recommendations and guidelines for producing maintainable and less buggy software (I am not talking about naming conventions and formatting here). For example MISRA-C and C++ contain a lot of recommendations that are result of a poor language design in the first place.
MISRA C/C++

Sure, but ... MISRA is *ONLY* about the code  :D

In avionics it's DO178B level { A .. E }, so it's MISRA integrated with ICE + Eng + test-report + QA, all applied to the whole software lifecycle.

Wiki pages and university courses can simply list concepts that you can also consider separately, it doesn't make any problem, but real life avionics has to consider all the specific know-how here and there, but all reflected among both developers and testers.

So, you understand WHY you have to write your C or Ada code in a specific way. Usually, because it's useful for your ICE-AI, and/or for testers, or something that is not *MISRA* specific, but actually helps a lot with the software life-cycle.

It's like The Matrix: talking about that is not like living it. You need to see yourself.

(o'man, am I really red-pilling? or something?
Why do I feel like Morpheus? Goshh  :o :o :o )
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 19, 2023, 02:54:36 pm
If you think about C and C++ languages for example, there are many recommendations and guidelines for producing maintainable and less buggy software (I am not talking about naming conventions and formatting here). For example MISRA-C and C++ contain a lot of recommendations that are result of a poor language design in the first place.
MISRA C/C++

Sure, but ... MISRA is *ONLY* about the code  :D

In avionics it's DO178B level { A .. E }, so it's MISRA integrated with ICE + Eng + test-report + QA, all applied to the whole software lifecycle.

Wiki pages and university courses can simply list concepts that you can also consider separately, it doesn't make any problem, but real life avionics has to consider all the specific know-how here and there, but all reflected among both developers and testers.

So, you understand WHY you have to write your C or Ada code in a specific way. Usually, because it's useful for your ICE-AI, and/or for testers, or something that is not *MISRA* specific, but actually helps a lot with the software life-cycle.

It's like The Matrix: talking about that is not like living it. You need to see yourself.

(o'man, am I really red-pilling? or something?
Why do I feel like Morpheus? Goshh  :o :o :o )

I wanted to limit my writings mostly into programming language-domain, as this thread is about a [new] programming language. Of course designing and implementing an embedded system is much more than just writing code with a specific programming language.

Since we are talking about a [new] programming language, I just wanted to list a few things that should take into a consideration when designing a new language. Of course the author of a programming language is free to make his/her own design decisions.

We have already 70+ years worth of experience of different programming languages. A new programming language should take the good parts of the existing languages, and leave out or fix the bad parts. It is not a very clever idea to repeat the errors and faults of existing and past programming languages, if they could be prevented in the first place with some careful planning.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 03:44:57 pm
I asked ChatGPT "Let's assume that you need to design a new programming language for embedded systems that will be targeted for safety-critical and mission-critical embedded systems, and provide safe memory handling and safe array operators. What would it be like?"
Quote
A programming language for embedded systems that is targeted for safety-critical and mission-critical systems would likely have the following features:

1. Memory safety: The language would have built-in mechanisms to prevent common memory-related errors such as buffer overflows, use-after-free, and null pointer dereferences.

2. Type safety: The language would have a strong type system that helps prevent type-related errors, such as type casting errors and implicit type conversions.

3. Array bounds checking: The language would have built-in array bounds checking to prevent out-of-bounds errors when accessing arrays.

4. Automatic memory management: The language would have a garbage collector or other mechanism for automatic memory management, which would help prevent memory leaks and other related errors.

5. Strict syntax and semantics: The language would have a strict syntax and semantics, in order to minimize the chances of introducing new errors.

6. Formal verification: The language would have formal verification tools that could be used to prove the absence of certain types of errors, such as data races.

7. Strong error handling: The language would have strong error handling, in order to prevent errors from propagating and causing further issues.

8. Support for concurrency: The language would have built-in support for concurrency, in order to make it easier to write multi-threaded code that is also safe.

9. Support for functional programming: The language would support functional programming features such as immutable variables, closures, and higher-order functions, which would help make the code more predictable and less error-prone.

10. Support for hardware-level control: The language would have support for interacting with low-level hardware, in order to allow for the precise control that is often required in embedded systems.

11. Support for real-time constraints: The language would have built-in support for real-time constraints, such as hard real-time and soft real-time, allowing developers to specify time constraints and deadlines for specific operations.

12. Interoperability: The language would have the ability to interoperate with other languages, such as C and C++, which are commonly used in embedded systems. This would allow for the use of existing libraries and codebases, and make it easier to integrate the new language with existing systems.

13. Standard libraries: The language would have a comprehensive set of standard libraries, including data structures and algorithms, for use in embedded systems.

14. Compact and efficient: The language would be designed to be compact and efficient, in order to minimize memory and processing requirements, which are important considerations in embedded systems.

15. Security features: The language would have built-in security features such as secure data handling and encryption to protect against common security threats.

16. Tooling and debugging: The language would have a robust set of tooling and debugging options, allowing developers to easily locate and fix errors.

17. Documentations: The language would have comprehensive and up-to-date documentation, including tutorials, guides, and reference material, to help developers get started quickly and become proficient with the language.

18. Community support: The language would have a strong and active community of developers, providing support and contributing to the language's development and maintenance.

Overall, the language would be designed with safety and reliability in mind, with a focus on preventing errors and ensuring that the code is predictable and easy to understand. It would also be efficient and flexible enough to meet the specific requirements of embedded systems.

19. Certification: The language would be designed to meet industry standards and certifications for safety-critical systems, such as ISO 26262 for automotive systems and DO-178C for avionics systems.

20. Safety-critical libraries: The language would have a set of libraries specifically designed for safety-critical systems that provide functionality such as fault-tolerance and redundancy.

21. Safety-critical constructs: The language would have constructs such as assertions and monitors that can be used to express safety properties and to check them at runtime.

22. Model-based development: The language would support model-based development, which allows for the design, verification, and validation of safety-critical systems using formal methods.

23. Traceability: The language would have traceability features that allow developers to trace requirements, design, and implementation, which is critical for safety-critical systems.

24. Auditing and logging: The language would have built-in support for auditing and logging which allows to check and trace the actions of the system and detect malfunctions that could cause safety hazards.

25. Testability: The language would be designed with testability in mind, allowing developers to easily create and execute unit tests, integration tests, and system tests to verify the correct functioning of the system.

Overall, the new programming language for embedded systems would be designed to meet the specific needs of safety-critical and mission-critical systems, providing a combination of safety, reliability, and efficiency, while at the same time meeting industry standards and certifications.

26. Design by Contract: The language would support Design by Contract (DbC) paradigm, which allows developers to express preconditions, postconditions, and class invariants that specify the expected behavior of the functions and classes, making it easier to understand and reason about the code.

27. Static analysis: The language would have built-in static analysis tools that can analyze the code and detect potential errors, such as null pointer dereferences, race conditions, and buffer overflows, before the code is even run.

28. Code review: The language would have tools and processes in place to support formal code reviews, where code is reviewed by other developers, in order to catch errors and improve the quality of the code.

29. Built-in safety mechanisms: The language would have built-in safety mechanisms such as watchdog timers, which can automatically reset the system if it becomes unresponsive, and fail-safe modes, which can be activated if a critical error is detected.

30. Support for formal verification: The language would support formal verification using tools such as model checkers, theorem provers, and static analyzers, which can be used to prove that the code meets certain safety properties.

31. Support for safety standards: The language would be designed to support safety standards such as IEC 61508 and ISO 26262, which provide guidelines for the development of safety-critical systems.

Overall, the new programming language for embedded systems would provide a comprehensive set of features for safety-critical and mission-critical systems, including memory safety, type safety, array bounds checking, automatic memory management, strict syntax and semantics, formal verification, strong error handling, support for concurrency, functional programming, and hardware-level control. Additionally, it would have features to support real-time constraints, interoperability, standard libraries, compact and efficient, security, tooling and debugging, documentation, community support and safety-critical libraries, constructs, model-based development, traceability, auditing and logging, testability, Design by Contract, static analysis, code review, built-in safety mechanisms, support for formal verification and safety standards.


This is a good list to keep in mind when designing a new programming language. Take existing programming languages used in safety-critical embedded systems (such as C, C++, Ada and Rust), implement their good features, and leave out and fix their problematic features. Also, make sure that the new programming language will meet requirements 5 and 18.

That's a superb list, very interesting indeed, I will indeed absorb this, exactly the kind of thing I'm looking for!

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 03:51:46 pm
Quote
Also, make sure that the new programming language will meet requirements 5 and 18.

This is a requirement I value hugely. I want to reduce implementation defined and undefined behaviors as much as possible. In that regard this has been noted as likely an essential feature: order of evaluation of arguments in procedure/function calls being strictly left-to-right (as done in Java and C# and perhaps other langauges).
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 19, 2023, 04:08:25 pm
I wanted to limit my writings mostly into programming language-domain, as this thread is about a [new] programming language. Of course designing and implementing an embedded system is much more than just writing code with a specific programming language.
...
We have already 70+ years worth of experience of different programming languages. A new programming language should take the good parts of the existing languages, and leave out or fix the bad parts. It is not a very clever idea to repeat the errors and faults of existing and past programming languages, if they could be prevented in the first place with some careful planning.

Precisely.

Your ChatGPT post illustrates the problems society is going to have in the near future. Currently ChatGPT seems to make at least as much sense as some of the posters on this forum :)

We know how poorly other conversational AI systems perform in real life, often allowing themselves to be perverted (and in at least case that word was very apt) by their training sets :(
Title: Re: The Imperium programming language - IPL
Post by: magic on January 19, 2023, 04:40:32 pm
I think ChatGPT gave pretty good answer, considering the fact it took only few minutes to get that list. I would consider the current state of AI as an assisting technology, and the user needs to be able to evaluate the answers for sanity, correctness and usefulness.
It gave you the answers you wanted to hear, which is its whole purpose ;)

The holy grail of marketing is here. But ask it to actually design something new and then :scared:

And, what would be your answer to that same question then? According to your reply, you have some other properties that are more important in a programming language.
Sure, here are my two most desirable features of any software product:

1. It should be possible to develop.
2. It should solve customer's problem.

This chatbot fails at it by generating a long laundry list of features which inevitably conflict with one another and having not even a slightest idea what to compromise and how to make something useful out of it. You won't have much memory safety with C interoperability and deep integration with legacy codebases, gonna need to find some way to carefully isolate these two worlds. Garbage collection may be tricky too, what if the C code stashes references to your objects? GC also complicates the requirements for compactness and efficiency and realtime guarantees.

Collecting a list of desirable features is the easiest 1% of the job of designing software, often already done for you by others. The hard part is knowing what can't be done, what must be compromised and how to realize customer's goals while staying within the annoying limits of reality. Also, the customer is usually having an XY problem and it's better to realize it sooner than later.

These chatbots are good at collecting and summarizing Internet memes relevant to your prompt, but that's all they really do. They sound smart because they repeat the same talking points you hear everyday and everywhere from everyone - they are getting very good at picking that up. But you could get about the same results by reading some Wikipedia article or starting a thread on the orange forum. These (plus writing marketing drivel) are likely the first human provided services which will get automated away by AI :P


And speaking of the concrete output from the bot, I am personally quite far from anything really safety critical, but here's my $.02:
- I don't think functional programming is common practice in embedded, but the bot picked it up because f.p. fanboys surely talk about it a lot
- formal verification is a hell difficult problem being worked on since the field's inception, show us your concrete ideas, please, because anyone can simply name-drop it
- not sure what "built-in encryption to protect against common security threats" means in a programming language
- what sort of "tools and processes" you need in a language to support code review?
- watchdog timers built into a programming language - I think it got carried away a little
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 19, 2023, 04:47:41 pm
Collecting a list of desirable features is the easiest 1% of the job of designing software, often already done for you by others.
The hard part is knowing what can't be done, what must be compromised and how to realize customer's goals while staying within the annoying limits of reality.
Also, the customer is usually having an XY problem and it's better to realize it sooner than later.

Thus spake someone with real-world experience :) Too true on all counts.
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 19, 2023, 05:01:35 pm
And speaking of the concrete output from the bot, I am personally quite far from anything really safety critical, but here's my $.02:
- I don't think functional programming is common practice in embedded, but the bot picked it up because f.p. fanboys surely talk about it a lot
- formal verification is a hell difficult problem being worked on since the field's inception, show us your concrete ideas, please, because anyone can simply name-drop it
- not sure what "built-in encryption to protect against common security threats" means in a programming language
- what sort of "tools and processes" you need in a language to support code review?
- watchdog timers built into a programming language - I think it got carried away a little

Yes, I decided to use raw, unedited ChatGPT output and leave those into the list* although they are not really related to a programming language per se. Like I wrote before: "I would consider the current state of AI as an assisting technology, and the user needs to be able to evaluate the answers for sanity, correctness and usefulness.".

Edit: *If someone runs the same phrase, ChatGPT would spit probably a similar kind of list those items in the list anyway.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 05:47:30 pm
Also, make sure that the new programming language will meet requirements 5 and 18.

 :D

Quote
18. Community support: The language would have a strong and active community of developers, providing support and contributing to the language's development and maintenance.

impossible, everyone in the community will soon be in the ignore list

===============================================

By the way, Even though there is no rotate operator in C, compilers that target actitectures that have rotate instructions usually recognize particular constructs and translate them into rotate instructions

Quote from: XC8 Compiler Manual
Rotate operations can be performed in your code, despite the C language not including a rotate operator.
The compiler will detect expressions that implement rotate operations using shift and logical operators and compile
them efficiently.
For the following code:
c = (c << 1) | (c >> 7);
if c is unsigned and non-volatile, the compiler will detect that the intended operation is a rotate left of 1 bit and will
encode the output using the PIC MCU rotate instructions. A rotate left of 2 bits would be implemented with code like:
c = (c << 2) | (c >> 6);
This code optimization will also work for integral types larger than a char. If the optimization cannot be applied, or
this code is ported to another compiler, the rotate will be implemented, but typically with shifts and a bitwise OR
operation.

or provide a builtin that map to the dedicated instruction (but you usually want to use the rotate in algorythms you are already writing in assembly sooooo.....)

I've only explicitly ignored posters who are demonstrably insulting. I don't mind someone disagreeing, in fact I value disagreement but only from someone I can respect, someone who argues impersonally, I really am surprised at the ease with which some people want to descend into abuse and bickering, I have no time for that.

Now regarding rotate operators. It is absolutely right to question whether these are a good or bad idea in a new language, there are numerous potential operators one can devise - especially given the rare freedom to define a language and grammar from scratch with no externally imposed constraints.

The way to do that is to accumulate a list of potential operators that can be implemented and then sift, filter that list perhaps, remove any dead wood and so on, but to say "this is just stupid" is not the way to handle such a challenge.

Now consider this that C++ has (can) offer (https://stackoverflow.com/questions/776508/best-practices-for-circular-shift-rotate-operations-in-c) "_rotl" and "_rotr" and consider that Microsoft also offer rotations as part of .Net in their BitOperations (https://learn.microsoft.com/en-us/dotnet/api/system.numerics.bitoperations?view=net-7.0) static class.

I simply took the step of advocating symbolic operators for these rather than intrinsics (which use method call syntax) for two reasons, 1. Operators are universal, not tied to a particular written/spoken language and 2. They are inherently "overloaded" so to speak, whereas method call syntax demands multiple methods for multiple argument types (unless one permits overloading in the language, which is a distinct possibility).

I do perform some degree of due diligence here, I don't make up an operator just because one can, I did it because it seemed logical, it complimented the shift operators, it lends symmetry to the language, whether they be worth keeping is an open question, but it isn't "stupid" to consider the possibilities.



Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 05:50:35 pm
Quote
18. Community support: The language would have a strong and active community of developers, providing support and contributing to the language's development and maintenance.

impossible, everyone in the community will soon be in the ignore list

This, was quite hilarious seeing the OP rage-ignoring the few world class experts who happen to hang on this forum ;D. (You know who you are, of course.)

I can totally see in my mind where OP's "experience" is coming from: working as a middle manager who simply fires everybody who says "no" to his absolutely stupid ideas, and collects a bunch of so called "yes men" ( https://www.urbandictionary.com/define.php?term=yes%20men (https://www.urbandictionary.com/define.php?term=yes%20men) ) around him. And there is no shortage of yes men.

This is the classic pattern of destroying the productivity of a business, ignoring those who do understand, and spending all the time discussing the color and construction materials of the bike shed and micromanaging this discussion. And the business can appear to still run, based on buying out smaller companies with their product lines which are profitable for some time before getting destroyed. And/or outsourcing projects that need to be done, to smaller companies with different mindset.

Here we go again, you actually want me to ignore you, another rude insulting rant (emphasis added by me) where you attack demons of your own making. You're ignored for abusive and demeaning  language.

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 06:13:53 pm
I asked ChatGPT "Let's assume that you need to design a new programming language for embedded systems that will be targeted for safety-critical and mission-critical embedded systems, and provide safe memory handling and safe array operators. What would it be like?"
Quote
A programming language for embedded systems that is targeted for safety-critical and mission-critical systems would likely have the following features:

1. Memory safety: The language would have built-in mechanisms to prevent common memory-related errors such as buffer overflows, use-after-free, and null pointer dereferences.

2. Type safety: The language would have a strong type system that helps prevent type-related errors, such as type casting errors and implicit type conversions.

3. Array bounds checking: The language would have built-in array bounds checking to prevent out-of-bounds errors when accessing arrays.

4. Automatic memory management: The language would have a garbage collector or other mechanism for automatic memory management, which would help prevent memory leaks and other related errors.

5. Strict syntax and semantics: The language would have a strict syntax and semantics, in order to minimize the chances of introducing new errors.

6. Formal verification: The language would have formal verification tools that could be used to prove the absence of certain types of errors, such as data races.

7. Strong error handling: The language would have strong error handling, in order to prevent errors from propagating and causing further issues.

8. Support for concurrency: The language would have built-in support for concurrency, in order to make it easier to write multi-threaded code that is also safe.

9. Support for functional programming: The language would support functional programming features such as immutable variables, closures, and higher-order functions, which would help make the code more predictable and less error-prone.

10. Support for hardware-level control: The language would have support for interacting with low-level hardware, in order to allow for the precise control that is often required in embedded systems.

11. Support for real-time constraints: The language would have built-in support for real-time constraints, such as hard real-time and soft real-time, allowing developers to specify time constraints and deadlines for specific operations.

12. Interoperability: The language would have the ability to interoperate with other languages, such as C and C++, which are commonly used in embedded systems. This would allow for the use of existing libraries and codebases, and make it easier to integrate the new language with existing systems.

13. Standard libraries: The language would have a comprehensive set of standard libraries, including data structures and algorithms, for use in embedded systems.

14. Compact and efficient: The language would be designed to be compact and efficient, in order to minimize memory and processing requirements, which are important considerations in embedded systems.

15. Security features: The language would have built-in security features such as secure data handling and encryption to protect against common security threats.

16. Tooling and debugging: The language would have a robust set of tooling and debugging options, allowing developers to easily locate and fix errors.

17. Documentations: The language would have comprehensive and up-to-date documentation, including tutorials, guides, and reference material, to help developers get started quickly and become proficient with the language.

18. Community support: The language would have a strong and active community of developers, providing support and contributing to the language's development and maintenance.

Overall, the language would be designed with safety and reliability in mind, with a focus on preventing errors and ensuring that the code is predictable and easy to understand. It would also be efficient and flexible enough to meet the specific requirements of embedded systems.

19. Certification: The language would be designed to meet industry standards and certifications for safety-critical systems, such as ISO 26262 for automotive systems and DO-178C for avionics systems.

20. Safety-critical libraries: The language would have a set of libraries specifically designed for safety-critical systems that provide functionality such as fault-tolerance and redundancy.

21. Safety-critical constructs: The language would have constructs such as assertions and monitors that can be used to express safety properties and to check them at runtime.

22. Model-based development: The language would support model-based development, which allows for the design, verification, and validation of safety-critical systems using formal methods.

23. Traceability: The language would have traceability features that allow developers to trace requirements, design, and implementation, which is critical for safety-critical systems.

24. Auditing and logging: The language would have built-in support for auditing and logging which allows to check and trace the actions of the system and detect malfunctions that could cause safety hazards.

25. Testability: The language would be designed with testability in mind, allowing developers to easily create and execute unit tests, integration tests, and system tests to verify the correct functioning of the system.

Overall, the new programming language for embedded systems would be designed to meet the specific needs of safety-critical and mission-critical systems, providing a combination of safety, reliability, and efficiency, while at the same time meeting industry standards and certifications.

26. Design by Contract: The language would support Design by Contract (DbC) paradigm, which allows developers to express preconditions, postconditions, and class invariants that specify the expected behavior of the functions and classes, making it easier to understand and reason about the code.

27. Static analysis: The language would have built-in static analysis tools that can analyze the code and detect potential errors, such as null pointer dereferences, race conditions, and buffer overflows, before the code is even run.

28. Code review: The language would have tools and processes in place to support formal code reviews, where code is reviewed by other developers, in order to catch errors and improve the quality of the code.

29. Built-in safety mechanisms: The language would have built-in safety mechanisms such as watchdog timers, which can automatically reset the system if it becomes unresponsive, and fail-safe modes, which can be activated if a critical error is detected.

30. Support for formal verification: The language would support formal verification using tools such as model checkers, theorem provers, and static analyzers, which can be used to prove that the code meets certain safety properties.

31. Support for safety standards: The language would be designed to support safety standards such as IEC 61508 and ISO 26262, which provide guidelines for the development of safety-critical systems.

Overall, the new programming language for embedded systems would provide a comprehensive set of features for safety-critical and mission-critical systems, including memory safety, type safety, array bounds checking, automatic memory management, strict syntax and semantics, formal verification, strong error handling, support for concurrency, functional programming, and hardware-level control. Additionally, it would have features to support real-time constraints, interoperability, standard libraries, compact and efficient, security, tooling and debugging, documentation, community support and safety-critical libraries, constructs, model-based development, traceability, auditing and logging, testability, Design by Contract, static analysis, code review, built-in safety mechanisms, support for formal verification and safety standards.


This is a good list to keep in mind when designing a new programming language. Take existing programming languages used in safety-critical embedded systems (such as C, C++, Ada and Rust), implement their good features, and leave out and fix their problematic features. Also, make sure that the new programming language will meet requirements 5 and 18.

At first pass these points strike me as being more fundamental than the others and much thought has gone into these too:

1. Memory safety: The language would have built-in mechanisms to prevent common memory-related errors such as buffer overflows, use-after-free, and null pointer dereferences.

Under that general heading I've included inability to "calculate" pointers/addresses, abolish pointer arithmetic. Also to include the "offset" as a type complimentary to "pointer" where an offset acts as a relative address rather than an absolute address, able to use the same -> notation though. Additional support fixed and varying length strings, no null terminated pseudo-strings.

3. Array bounds checking: The language would have built-in array bounds checking to prevent out-of-bounds errors when accessing arrays.

Here I think we can support two modes, fast and slow. The fast simply ensures that the referenced element lies inside the array, it might be accessed with illegal subscripts, be the wrong element, but it is inside the array. The slow verifies each subscript against the arrays defined bounds. In a 10x10 array for example accessing [2,12] is wrong but won't corrupt system memory (only user data memory). These checks are not hard to do and some CPUs even offer instructions to assist. This capability could also be enabled/disabled at runtime, perhaps even on a per-array basis...

5. Strict syntax and semantics: The language would have a strict syntax and semantics, in order to minimize the chances of introducing new errors.

Well the syntax is pretty formal, derived from PL/I which was more formal, more systematic than C and its derivatives. For example the grammar contains no ambiguities unlike C and C++ which can greatly complicate parsing, stuff like "X * Y;" is common in C and C++ but a parser cannot tell if it's a declaration of a pointer to a type X or a multiplication, it needs to do semantic analysis as part of parsing (i.e. these are not context free languages so cannot be parsed by context free parsing), this is undesirable and so grammars based on C have been rejected.

7. Strong error handling: The language would have strong error handling, in order to prevent errors from propagating and causing further issues.

Exception support is needed here. Again this first appeared as a language capability in PL/I, (where they called it "conditions" PL/I was written by expert IBM mainframe assembler programmers and that has a "condition code register" but I digress). So yes, this is a must IMHO.

10. Support for hardware-level control: The language would have support for interacting with low-level hardware, in order to allow for the precise control that is often required in embedded systems.

This needs better definition IMHO. The language does offer "bit" as a data type and supports declaring arrays of bits and various alignment options too. But is there more too this? What does "low level" really mean? I can see scope for abstract access to the stack, some operations that might allow stack walking perhaps but we soon get to target specific stuff which is undesirable.

27. Static analysis: The language would have built-in static analysis tools that can analyze the code and detect potential errors, such as null pointer dereferences, race conditions, and buffer overflows, before the code is even run.

This is good, I've not thought much about this though, so this needs more definition perhaps...

Thoughts???

Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 06:52:22 pm
Quote
I've only explicitly ignored posters who are demonstrably insulting

demonstrably insulting, an insult to the insult itself  :o :o :o
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 07:32:18 pm
Quote
Here we go again, you actually want me to ignore you, another rude insulting rant (emphasis added by me) where you attack demons of your own making. You're ignored for abusive and demeaning  language.

So, this dude really *blames others* for his social and tecnical defects.
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 19, 2023, 07:42:53 pm
Now that ChatGPT has been involved in this process, I think we're gonna reach the sublime!
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 19, 2023, 08:22:42 pm
sublime!

talking about sublime, I checked an old copy of my Dr. Dobb's Journal, and it says ROL and ROR were useful during i386 era for bit-by-bit examining a value in a way that is non-information-destructive.

Kind of bit(x)-test for intel chips :o :o :o

Then it points out something like this (https://blog.regehr.org/archives/1063). Nice, isn't it?
("Safe, Efficient, and Portable Rotate in C/C++")

Code: [Select]
uint32_t rotl32a (uint32_t x, uint32_t n)
{
  return (x<<n) | (x>>(32-n));
}
That's basically where what @Jportici wrote came from :o :o :o

(yep, today I am in the mood of doing computer archeology
arche-comput-ology, or something)
Title: Re: The Imperium programming language - IPL
Post by: Kalvin on January 19, 2023, 08:39:11 pm
3. Array bounds checking: The language would have built-in array bounds checking to prevent out-of-bounds errors when accessing arrays.

Here I think we can support two modes, fast and slow. The fast simply ensures that the referenced element lies inside the array, it might be accessed with illegal subscripts, be the wrong element, but it is inside the array. The slow verifies each subscript against the arrays defined bounds. In a 10x10 array for example accessing [2,12] is wrong but won't corrupt system memory (only user data memory). These checks are not hard to do and some CPUs even offer instructions to assist. This capability could also be enabled/disabled at runtime, perhaps even on a per-array basis...

When passing an array to a function, the compiler should also pass the the length of the array to the function, so that the function can obtain the length of the array at the run-time.

For example, in C this needs to be done manually by introducing an additional parameter to the function. Also, any array bounds checking needs to be done manually. It is very easy to produce off-by-one array access errors in C.

The programmer should be able to enable/disable the compiler-generated array bounds checking. During unit testing, the programmer may want to enable all checks for best test coverage. For the production code, the programmer may want to disable the bounds checking selectively in some parts of the code.

10. Support for hardware-level control: The language would have support for interacting with low-level hardware, in order to allow for the precise control that is often required in embedded systems.

This needs better definition IMHO. The language does offer "bit" as a data type and supports declaring arrays of bits and various alignment options too. But is there more too this? What does "low level" really mean? I can see scope for abstract access to the stack, some operations that might allow stack walking perhaps but we soon get to target specific stuff which is undesirable.

I read this as "the programming language should have means to define and access the peripheral registers with an ease".

Typically the operations for peripheral registers are reading/writing a register, and testing/setting/clearing/toggling a bit or set of bits in the register.

The language should have support for defining bits and bitmasks for the registers, and prevent using a (wrong) bit mask defined for another register.

This whole register access-topic is quite complex, and needs careful analysis to get it right. For example, some registers may be read-only, some write-only, and some register may contain bits that are read-only, etc. It would be a good thing if the programming language had support for expressing these things, and the compiler could check for proper register and bit accesses. At this point I do not have any good ideas for this, though.

27. Static analysis: The language would have built-in static analysis tools that can analyze the code and detect potential errors, such as null pointer dereferences, race conditions, and buffer overflows, before the code is even run.

This is good, I've not thought much about this though, so this needs more definition perhaps...

This might be tricky, and I know very little about this topic. However, using compile-time static asserts/checks, and checks/asserts during runtime with preconditions, post-conditions and other checks could help conveying the information how the functions are expected to behave and what kind of parameter values functions are expecting. These run-time checks can be enabled during unit testing, and selectively turned off in the production code. The compiler should be able to ascertain that there are no unwanted side effects when performing these checks.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 08:41:02 pm
Comparison Chaining

I've been reading about how some languages support comparison operator chaining. Now the grammars of most languages support it, it is a semantic question really.

Code: [Select]

if a < b < c > d then
   log("Conditions Met.");


This can be interpreted as

Code: [Select]

if a < b && b < c && c > d then
   log("Conditions Met.");


I find this quite appealing, thoughts?



Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 08:51:26 pm
3. Array bounds checking: The language would have built-in array bounds checking to prevent out-of-bounds errors when accessing arrays.

Here I think we can support two modes, fast and slow. The fast simply ensures that the referenced element lies inside the array, it might be accessed with illegal subscripts, be the wrong element, but it is inside the array. The slow verifies each subscript against the arrays defined bounds. In a 10x10 array for example accessing [2,12] is wrong but won't corrupt system memory (only user data memory). These checks are not hard to do and some CPUs even offer instructions to assist. This capability could also be enabled/disabled at runtime, perhaps even on a per-array basis...

When passing an array to a function, the compiler should also pass the the length of the array to the function, so that the function can obtain the length of the array at the run-time.

For example, in C this needs to be done manually by introducing an additional parameter to the function. Also, any array bounds checking needs to be done manually. It is very easy to produce off-by-one array access errors in C.

The programmer should be able to enable/disable the compiler-generated array bounds checking. During unit testing, the programmer may want to enable all checks for best test coverage. For the production code, the programmer may want to disable the bounds checking selectively in some parts of the code.

10. Support for hardware-level control: The language would have support for interacting with low-level hardware, in order to allow for the precise control that is often required in embedded systems.

This needs better definition IMHO. The language does offer "bit" as a data type and supports declaring arrays of bits and various alignment options too. But is there more too this? What does "low level" really mean? I can see scope for abstract access to the stack, some operations that might allow stack walking perhaps but we soon get to target specific stuff which is undesirable.

I read this as "the programming language should have means to define and access the peripheral registers with an ease".

Typically the operations for peripheral registers are reading/writing a register, and testing/setting/clearing/toggling a bit or set of bits in the register.

The language should have support for defining bits and bitmasks for the registers, and prevent using a (wrong) bit mask defined for another register.

This whole register access-topic is quite complex, and needs careful analysis to get it right. For example, some registers may be read-only, some write-only, and some register may contain bits that are read-only, etc. It would be a good thing if the programming language had support for expressing these things, and the compiler could check for proper register and bit accesses. At this point I do not have any good ideas for this, though.

27. Static analysis: The language would have built-in static analysis tools that can analyze the code and detect potential errors, such as null pointer dereferences, race conditions, and buffer overflows, before the code is even run.

This is good, I've not thought much about this though, so this needs more definition perhaps...

This might be tricky, and I know very little about this topic. However, using compile-time static asserts/checks, and checks/asserts during runtime with preconditions, post-conditions and other checks could help conveying the information how the functions are expected to behave and what kind of parameter values functions are expecting. These run-time checks can be enabled during unit testing, and selectively turned off in the production code. The compiler should be able to ascertain that there are no unwanted side effects when performing these checks.

Speaking of array passing, I agree and PL/I offered all of this and more from the outset. It's a capability long overlooked yet curiously was made available in the 60s.

Any array whether it has bounds known at compile time or not until runtime, has (under the hood) a array "descriptor", metadata that describes everything about an array, any array, like:

1. Element size for use in subscript/offset calculations
2. Rank
3. Upper/Lower bound of each dimension

This can be stored as part of the memory allocated for the array (be it stack or heap). This then makes it trivial for executing code to get at this metadata using language builtin functions.

Additionally we can pass arrays to code that is "generic" by having that code declare the arrays as:

Code: [Select]

procedure sort_table (table);

   arg table(*,*,*) bin(15);

end;


that code "sort_table" can then at runtime discover the bounds of the array, different parts of the code can pass different sized arrays (but they must all be 3D).

So this is absolutely going to present, non negotiable. The use of * is very different in PL/I and C, so that can confuse some (it's original use back in the day was "wildcard").

A similar thing holds for strings, both fixed and variable length strings can also be declared as * args in code and the callee able to discern the metadata at runtime whatever length strings are actually passed.


Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 19, 2023, 09:17:13 pm
sublime!

talking about sublime, I checked an old copy of my Dr. Dobb's Journal, and it says ROL and ROR were useful during i386 era for bit-by-bit examining a value in a way that is non-information-destructive.

Kind of bit(x)-test for intel chips :o :o :o

Then it points out something like this (https://blog.regehr.org/archives/1063). Nice, isn't it?
("Safe, Efficient, and Portable Rotate in C/C++")

Code: [Select]
uint32_t rotl32a (uint32_t x, uint32_t n)
{
  return (x<<n) | (x>>(32-n));
}
That's basically where what @Jportici wrote came from :o :o :o

I'm sure it was well known long before Dr Dobbs.

With the definition of shifts used in RISC-V and others you don't even need the word-size dependent "32-n", just "-n" works fine.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 19, 2023, 09:20:49 pm
3. Array bounds checking: The language would have built-in array bounds checking to prevent out-of-bounds errors when accessing arrays.

Here I think we can support two modes, fast and slow. The fast simply ensures that the referenced element lies inside the array, it might be accessed with illegal subscripts, be the wrong element, but it is inside the array. The slow verifies each subscript against the arrays defined bounds. In a 10x10 array for example accessing [2,12] is wrong but won't corrupt system memory (only user data memory). These checks are not hard to do and some CPUs even offer instructions to assist. This capability could also be enabled/disabled at runtime, perhaps even on a per-array basis...

When passing an array to a function, the compiler should also pass the the length of the array to the function, so that the function can obtain the length of the array at the run-time.

For example, in C this needs to be done manually by introducing an additional parameter to the function. Also, any array bounds checking needs to be done manually. It is very easy to produce off-by-one array access errors in C.

The programmer should be able to enable/disable the compiler-generated array bounds checking. During unit testing, the programmer may want to enable all checks for best test coverage. For the production code, the programmer may want to disable the bounds checking selectively in some parts of the code.

10. Support for hardware-level control: The language would have support for interacting with low-level hardware, in order to allow for the precise control that is often required in embedded systems.

This needs better definition IMHO. The language does offer "bit" as a data type and supports declaring arrays of bits and various alignment options too. But is there more too this? What does "low level" really mean? I can see scope for abstract access to the stack, some operations that might allow stack walking perhaps but we soon get to target specific stuff which is undesirable.

I read this as "the programming language should have means to define and access the peripheral registers with an ease".

Typically the operations for peripheral registers are reading/writing a register, and testing/setting/clearing/toggling a bit or set of bits in the register.

The language should have support for defining bits and bitmasks for the registers, and prevent using a (wrong) bit mask defined for another register.

This whole register access-topic is quite complex, and needs careful analysis to get it right. For example, some registers may be read-only, some write-only, and some register may contain bits that are read-only, etc. It would be a good thing if the programming language had support for expressing these things, and the compiler could check for proper register and bit accesses. At this point I do not have any good ideas for this, though.

27. Static analysis: The language would have built-in static analysis tools that can analyze the code and detect potential errors, such as null pointer dereferences, race conditions, and buffer overflows, before the code is even run.

This is good, I've not thought much about this though, so this needs more definition perhaps...

This might be tricky, and I know very little about this topic. However, using compile-time static asserts/checks, and checks/asserts during runtime with preconditions, post-conditions and other checks could help conveying the information how the functions are expected to behave and what kind of parameter values functions are expecting. These run-time checks can be enabled during unit testing, and selectively turned off in the production code. The compiler should be able to ascertain that there are no unwanted side effects when performing these checks.

The whole MCU register stuff is very interesting and I suspect there's room for innovation here. I was writing stuff recently on an STM32F4 board that uses SPI to interact with an attached NRF24 device. One of the bits in one of the NRF's "registers" (the status register if I recall) has a bit TX_DS and the act of writing a 1 to it sets it off!

Also many of the registers have several bit patterns that are illegal/unsupported, so somehow abstracting these kinds of things might be helpful too.

I actually wrote a lot of fiddly C to get this coded neatly, my experiences is written in this documentation (https://github.com/Steadsoft/embedded/tree/main/NucleoF446RE/NRF24/Transmitter#readme), it was while working on that that I began to get frustrated that the language and tools just were not a capable as I wanted them to be, got me thinking about how to address these fiddly C limitations.

Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 19, 2023, 09:40:06 pm
sublime!

talking about sublime, I checked an old copy of my Dr. Dobb's Journal, and it says ROL and ROR were useful during i386 era for bit-by-bit examining a value in a way that is non-information-destructive.

Kind of bit(x)-test for intel chips :o :o :o

Then it points out something like this (https://blog.regehr.org/archives/1063). Nice, isn't it?
("Safe, Efficient, and Portable Rotate in C/C++")

Code: [Select]
uint32_t rotl32a (uint32_t x, uint32_t n)
{
  return (x<<n) | (x>>(32-n));
}
That's basically where what @Jportici wrote came from :o :o :o

I'm sure it was well known long before Dr Dobbs.

Yeah. It's pretty basic. Of course it works not just with digits, but with any kind of objects in some order. So any kid with a decent IQ can probably devise that. ;D
Title: Re: The Imperium programming language - IPL
Post by: JPortici on January 20, 2023, 10:10:37 am
Now that ChatGPT has been involved in this process, I think we're gonna reach the sublime!

I wonder if we can let ChatGPT design a programming language
Title: Re: The Imperium programming language - IPL
Post by: JPortici on January 20, 2023, 10:12:34 am
Comparison Chaining

I've been reading about how some languages support comparison operator chaining. Now the grammars of most languages support it, it is a semantic question really.

Code: [Select]

if a < b < c > d then
   log("Conditions Met.");


This can be interpreted as

Code: [Select]

if a < b && b < c && c > d then
   log("Conditions Met.");


I find this quite appealing, thoughts?

it may be that i use too much C but i do not want to think about precedences, even when writing other languages.
So i use parentheses
I probably still use this notation only when writing on paper
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 20, 2023, 03:44:56 pm
Comparison Chaining

I've been reading about how some languages support comparison operator chaining. Now the grammars of most languages support it, it is a semantic question really.

Code: [Select]

if a < b < c > d then
   log("Conditions Met.");


This can be interpreted as

Code: [Select]

if a < b && b < c && c > d then
   log("Conditions Met.");


I find this quite appealing, thoughts?

it may be that i use too much C but i do not want to think about precedence, even when writing other languages.
So i use parentheses
I probably still use this notation only when writing on paper

Yes, I don't use it much myself but I was reading about the language Raku which has a huge number of operators and this is something that language offers (initially derive from Perl. Also do note, the above expression (a boolean valued expression) only involves comparison operators which all have identical precedence.

I don't want to abandon stuff that's deeply entrenched just because one can, for example I wondered whether using <= for assignment was worth considering:

Code: [Select]

dcl counter bin(15);

counter <= 123;

if counter = 321 then
   return;


That way the = becomes purely a comparison operator and assignment (which should never have even used = really) is visually distinct. This is extremely easy to do but the use of = for assign is so deeply entrenched that its of little value to introduce <= for this.

But in this case if you think about what "if (a < b < c > d)" means today in say C or C++ or Java or C#, Javascript etc it is not that useful anyway, how many of us would ever code a comparison that looks like that?

So adopting the chaining might be a net gain, the very few cases where the syntax might be used in C etc today are so few and so unusual that the gain of being able to condense all these chained compares might actually be worth it...

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 20, 2023, 07:51:40 pm
OK namespaces have been added to the grammar now, never part of the PL/I language or C, but grammatically almost trivial to add and to parse.

A namespace can contain statement blocks or other namespaces and a source fine can contain multiple (that is, not nested) namespaces too:

Code: [Select]
namespace test;

    dcl root_ptr pointer;

    procedure outer (arg);

       dcl counter bin(15);
       dcl window bin(31);

        if counter = 123 then
           return;
        end;

    end procedure;

    procedure rerun (arg);

        call s$system.memory.allocate_virgin(10 0000:H); // 1,048,576 in dec FYI

    end procedure;

end;

namespace utility.sundry;
   
    // an empty namespace

end;

namespace s$system;

    namespace memory;

        procedure allocate_virgin(size);

            arg size bin(31);

        end;
    end;
end;

The beauty is of course that if compiling multiple source files, the various procs/funcs all "accumulate" into their respective containing namespaces.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 20, 2023, 07:53:30 pm
Something that hasn't come up is "enums" this is often a subject that elicits many opinions, so I'd love to hear them.

PL/I never had enums, so adding these is pretty much green field.

Title: Re: The Imperium programming language - IPL
Post by: alexanderbrevig on January 20, 2023, 11:01:26 pm
Moderators: please can I stop notifications from this to thread? I've learned my lesson.
Title: Re: The Imperium programming language - IPL
Post by: Nominal Animal on January 21, 2023, 12:06:25 am
Moderators: please can I stop notifications from this to thread? I've learned my lesson.
See the buttons at the top right and bottom right of the entire thread: Reply, Notify/Unnotify, Mark unread, Send this topic, Print, Search?
If you see Notify, click on it, click Yes, and then it changes to Unnotify.  Click Unnotify, and Yes, and no more notifications!
(Even if new posts are made to this thread, it will not show up in the New Replies to Your Posts list.)

Edited: See my reply #947 (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4653283/#msg4653283) for the corrected instructions.

The very useful Ignore list is accessible from the Profile button near the top of every page (your own profile, that is), and the Summary subsection.  Click that Summary.  In the Profile Info page, just above the Summary bar, there are three buttons: Profile Info (which is highlighted), Modify Profile, and Actions.  If you click Modify Profile, go down to the Buddies/Ignore list... and click on Edit Ignore List, you can add user names (like "Nominal Animal" or "Sherlock Holmes") to your ignore list, so you don't need to see their posts: they will just be minimised with a small link saying "You are ignoring this user." with "Show me the post" link right next to it, so you can expand and see the post if you actually do want to read the post.

Personally, I use the ignore list whenever I believe I cannot respond in an useful manner to some member.  I can be quite ornery/grumpy/sensitive about certain patterns, especially when I perceive someone is misleading others and refuses to correct their mis-advice because it would hurt their ego.  I also like to argue/debate objectively measurable things, instead of opinions, and that can look like trolling or "wanting to have the final word".  And my posts tend to be looooong, too.  So, I can understand why some members will want to ignore my own posts, too.

I tend to trim/flush the list after a month or so, though, and currently have six members in my ignore list.  I do not want to stop them from posting, it's just that their worldview/understanding/advice is so different to mine that I do not believe we can interact in a mutually beneficial manner.  Plus, in threads in e.g. Beginner forums, I can always open and read their posts and even thank them if they help others, to show that it is not their person I object to, only their output in other domain-specific threads, and still appreciate it when they help others.  This "output, not the person" thing is particularly important to me, because it is the reason I've used this pseudonym for the last decade or so.

The best way –– and I often fail to do this myself :-[ –– is to use the ignore list quetly, and later on, if one sees their posts quoted by another member with useful information, one can always revise ones opinion.  In the mean time, not having to read their drivel (or in my own case, my long-ass walls of text), will reduce any social friction and unnecessary arguments on the forum.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 21, 2023, 04:07:06 pm
Moderators: please can I stop notifications from this to thread? I've learned my lesson.

Utterly astonishing remark from a purported creative, artistic mind.

 :-//

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 21, 2023, 04:27:35 pm
This is a start on enums:

Code: [Select]
     def devices enum bin(15),
          gps = 10,
          adc,
          spi
      end;

      def folders enum string(32),
          devices = "devices",
          logs = "logs,
          dumps = "dumps"
      end;

      def rates enum,
           fast,
           medium,
           slow,
           stopped
      end;

A type specifier after the "enum" keyword is optional. An explicit assigned value resets the way later values get set, so in example 1 "adc" would be set by the compiler, to 11.

This syntax is pretty straightforward with a starting "def" (or "define" if one wants to avoid abbreviated keywords) then an identifier followed by attributes.

PL/I never had user defined types so defining enums and structures (as a type) was never factored into the grammar, this has implications which I'm now trying to clarify.

Now we have a true and interesting grammar challenge, one of the more interesting aspects of language design.

If we permit user defined types (which this language absolutely must) then we get a problem with the original PL/I grammar.

Code: [Select]

dcl mydata folders; // dcl an instance of the "folders" enum


The problem here is we lose backward compatibility. The current grammar lets us add new keywords over time, so we might want to add a new language attribute "folders" nothing to do with the above user's enum defintion.

But that would cause a problem, code like the above declaration will fail to compile identically, because it has a user defined name "folders" and that will now be interpreted as a use of the new "folders" attribute.

As the grammar stands now we can easily create identifiers named "binary" or "decimal" or "string" and never have a problem even though these are language keywords for predefined types.

The syntax for declarations is basically

Code: [Select]
<dcl-keyword> <identifer> [multiple other attributes in any order] <semicolon>
With user defined types we are introducing a second option for having an additional <identifier> in the declaration.

One way to fix this is to have another keyword, not use "dcl", any kewyord we add can then have its own syntax defined, as arbitrarily rich as we like and no backward compatibility issue.

So perhaps that's the best way...


Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 21, 2023, 07:01:39 pm
Well this is now neatly resolved, and in actual fact PL/I provided the solution already, although it wasn't clear at the time.

Types are created with a new "type" keyword, a type can define an enum or a structure (and perhaps other ideas too).

Then there's another variant of the declare statement using "as":

Code: [Select]
type baud_rates enum string(31);

dcl rates as baud_rates;

This solves it, no user defined type's name can every clash with some future added keyword.

Of course the full syntax of enum and structure aren't in place above, but this is a formality, the original problem is solved.

PL/I does have a "like" keyword and was used to declare structures that shared the layout of another structure, I just used "as" instead.

This kind of thing parses fine now:

Code: [Select]
      type baud_rates enum bit(8),
           first  = 1010 0011:b,
           second = 1110 1101:b,
           third  = 1101 0111:b,
           fourth = 0010 1100:b
      end;




Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 21, 2023, 08:24:13 pm
Moderators: please can I stop notifications from this to thread? I've learned my lesson.
See the buttons at the top right and bottom right of the entire thread: Reply, Notify/Unnotify, Mark unread, Send this topic, Print, Search?
If you see Notify, click on it, click Yes, and then it changes to Unnotify.  Click Unnotify, and Yes, and no more notifications!
(Even if new posts are made to this thread, it will not show up in the New Replies to Your Posts list.)

No, that does not work. It controls notification emails or something, not "Show new replies to your posts", which I believe it is impossible to disable for a given thread.
Title: Re: The Imperium programming language - IPL
Post by: ataradov on January 21, 2023, 08:34:04 pm
There is a way to ignore topics. I have no idea how to activate it though. In the forum profile there is a section for ignored topics and I have one topic ignored there and it works, I see no mentions of it anywhere. I think it is activated from the unread replies page (https://www.eevblog.com/forum/unreadreplies/?start=0 (https://www.eevblog.com/forum/unreadreplies/?start=0)), but you actually need to have some replies there before you can do this.

Edit: Yes in that page once the topic has new replies you can select the checkbox and click ignore button. I just tried after magic's reply and it worked. Unignore back in the profile.

The only thread I ignore is TEA one, since there is too much traffic there and it just does not stop.
Title: Re: The Imperium programming language - IPL
Post by: magic on January 21, 2023, 08:47:49 pm
It's quite arcane. I have never done it myself, but I recall somebody saying that you need to first enable "quick moderation → as checkboxes" in forum profile and then the option for ignoring topics is supposed to appear.

I don't bother, I ignore topics, posts, users etc with the scroll wheel ;)
Title: Re: The Imperium programming language - IPL
Post by: Nominal Animal on January 21, 2023, 10:14:39 pm
No, that does not work. It controls notification emails or something, not "Show new replies to your posts", which I believe it is impossible to disable for a given thread.
You're right.  I mixed the darn things up!

Near the top of the page, click on Profile > Summary.
In the new page, just below that, click on Modify profile > Look and Layout.

If you are ignoring users, make sure the Hide messages posted by members on my ignore list is checked.

At the bottom of the list, for Show quick moderation as select Checkboxes.

Then, you need to wait until new replies appear in threads you want to ignore.
At the top of the page, you can see the Show new replies to my posts link; this leads to the Updated topics (https://www.eevblog.com/forum/unreadreplies/) page (for yourself); you can bookmark this if you want to check for replies to your posts.

This page lists threads with new posts.  At the right edge at the top of the topic list, you have three buttons, with the rightmost being Ignore topics.
Add a Checkmark at the right edge for each topic you want to ignore, and then click Ignore topics.
Yes, it is unfortunate that you need to wait until new replies are posted to that thread before you can ignore it, but it does work.
Title: Re: The Imperium programming language - IPL
Post by: MIS42N on January 22, 2023, 01:25:02 am
One of the things that is messy in assembler is using fractions in fixed point arithmetic. Floating point is fine when it is part of the instruction set, but not every microprocessor has FP.

So being able to define something as (in an arbitrary language) binary [8\8] would be nice, an 8 bit integer part and an 8 bit fraction part. Or [0\24], [16\16] etc. The code required is simple, just shift bits or add zero bytes before computation. But interpreting what goes on is not so clear. Being able to express it in the language would be helpful.

Is this already a thing, or is it too obscure to be worried about? Languages I've encountered assume int is integer (funny about that) and if fractions are thing, it is done in floating point with a floating point library if not native to the architecture.



Title: Re: The Imperium programming language - IPL
Post by: Nominal Animal on January 22, 2023, 03:54:38 am
So being able to define something as (in an arbitrary language) binary [8\8] would be nice, an 8 bit integer part and an 8 bit fraction part. [...] Is this already a thing, or is it too obscure to be worried about?
I have implemented something like that in Python, deriving from tuple.  Arithmetic, unary, and binary operators are just syntactic sugar on top of special methods (https://docs.python.org/3/reference/datamodel.html#emulating-numeric-types), so that e.g. a*b is equivalent to a.__mul__(b) (or if that is NotImplemented, then b.__rmul__(a)).  (I called dividing the integral numerator and the denominator (n/d) of a number, "normalization"; with divisor always positive, and only the numerator signed.  You end up doing this basically after every operation, which makes arithmetic using such a the type surprisingly "slow".  For Python, it doesn't matter much, because it is quite slow anyway.)
If I recall correctly, I used it for prime decomposition or something like that; a simple algorithmic solution to something, closer to HPC, and not really anything I'd need on a small microcontroller.

On the other hand, a very close cousin, interval arithmetic (https://en.wikipedia.org/wiki/Interval_arithmetic?useskin=vector) can be very useful even on small microcontrollers (and with both integer and floating-point math).  Essentially, each number is represented by an interval, min..max.  Operations like multiplication and division yield the largest interval when applied to the arguments.

(If you continue down that rabbit hole, you can represent each number by their probability distribution, for example their Gaussian probability distribution, so that the result of the same arithmetic operations you apply to scalar numbers, yields the probability distribution of the results when all inputs and constants are also specified as probability distributions.  It can be quite useful when dealing with uncertain and fuzzy data.  You will want to choose your basis distribution carefullt, though, and compose all other distributions as sums or variants of that.  Gaussian itself is a good choice, but there are others as well.)

I am not in favour of generic operator overload or operator definition, in the sense of how e.g. C++ uses >> and << with streams, but being able to define the functions corresponding to arithmetic, binary, and unary operators, is definitely useful with custom numeric types (fixed point formats, interval arithmetic, etc.) as well as vectors and matrices (and linear algebra in general).  Exactly how this is done depends on your language and preferences, of course, but I do prefer most the model where the operator function is solely determined by the types of the arguments to the operator.  For built-in arithmetic types (number types), the functions would be provided by the compiler.  For custom types, you'd define one function per type pair (for arithmetic and binary operators) or one function per type (for unary operators).  For example:
   dfn operator_multiply(int a, vec2d b): vec2d(a*b.x, a*b.y);
   dfn operator_multiply(vec2d a, int b): vec2d(a.x*b, a.y*b);
   dfn operator_multiply(vec2d a, vec2d b): int(a.x*b.y - a.y*b.x);
   dfn operator_divide(vec2d a, int b): vec2d(a.x/b, a.y/b);
   dfn operator_divide(int a, vec2d b): forbidden;

For optimization, it is quite important that the language has a concept of "pure" functions: functions whose return value depends only on their arguments, and have no side effects, so that they can be evaluated at any point and always yield the same result – and that calls to such functions can be freely rearranged, cached, and even omitted.  (The effect of such a concept for a compiler is easy to see, for example in Fortran 95 'forall' loops.  Also, GCC and Clang implement this even for C via the const function attribute (https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-const-function-attribute), __attribute__((const)).)



What I wonder myself, is whether developers would be confused with different "kinds" of functions –– like operators above, or say accessors that can only access their own object, but are otherwise "pure"/"const".  If you look and ask around, it is clear that even the distinction between Fortran functions and subroutines (the former returning a value, the latter not returning anything) makes people ask questions.  Would it be better to have separate function 'kinds', or just have attributes specifying their expected (and compile-time verified) properties?

I ended up in details like how to deal with say "pure" function when supplied with a parameter of a "volatile" type.  You can solve it at the model level (ie. "operator functions will have their arguments passed by value" in which case the volatility of the value nullifies reuse but makes the same function "work"), or you can have a way to state parameter type requirements, somewhat akin to old K&R C function or Fortran 95 function or subroutine parameter definition.

The rabbit hole went so far for me that I looked into the possibility of having the object file format support the definition (down to which register a parameter is passed in, as well as its type) of each parameter.  At some point it gets too far, as in requiring completely new file formats for the toolchain to work on, and not just a completely new toolchain...  ::)
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 22, 2023, 04:32:42 am
At the bottom of the list, for Show quick moderation as select Checkboxes.

Wow! I got the checkboxes. And have now just told the system to ignore this thread. There will be one or two other threads, often years-old, that get the same treatment next time they pop up. And probably a lot of new uninteresting threads on a daily basis, to stop them continually popping up in my "Show unread posts since last visit" list during their lifetime as it appears this feature is also available there. I might now stop ignoring some entire "boards".

That is one well-hidden feature!

Goodbye cruel thread!
Title: Re: The Imperium programming language - IPL
Post by: MIS42N on January 22, 2023, 05:06:07 am
The rabbit hole went so far for me that I looked into the possibility of having the object file format support the definition (down to which register a parameter is passed in, as well as its type) of each parameter.  At some point it gets too far, as in requiring completely new file formats for the toolchain to work on, and not just a completely new toolchain...  ::)
If the concept is workable, then difficulty of implementing it shouldn't be a barrier. A toolchain is written once (conceptually) but used millions of times. If it saves one millionth of the time it takes to create it, bonus. I may have mentioned (if not, I thought it) that letting the compiler decide just how a function/subroutine gets incorporated into final code (i.e pass a value, use a pointer, put it in a register, incorporate the code inline rather than as a callable (at the binary level) module, would improve code efficiency, and the source just gives preferences as to how it happens - fastest speed, minimum code, don't care.

It would require a rethink of the whole object library concept with library modules having flexible interfaces not resolved until link time, which in turn would need the compiler to be aware of how flexible it can be in managing those interfaces.

It could extend to libraries hinting at resource use rather than specifying. On a CPU like ATmega328P there are 32 registers, the C compiler (that I used) avoids using some registers because the libraries that may be incorporated use some of them by convention. But if the compiler was able to know which library components are used, and those components said something like"I need 3 registers but don't care which" then the compiler could optimise its use of registers.

I had not seen the term interval arithmetic before although I think I know the concept well. Maybe a tool? - I have these variables, this is the function, I want this accuracy in the output, what size intermediates are required? - actually no, it would have to be incorporated in the compiler as it would be creating intermediate values not in the source code. A=(1-X^2)/(1-Y^2) would have a few intermediate values, fine in floating point but more information needed to do it in fixed point.

interesting.
Title: Re: The Imperium programming language - IPL
Post by: Nominal Animal on January 22, 2023, 06:34:32 am
Goodbye cruel thread!
;D

But really, our –– those of us still participating –– loss. :'(

[...] having the object file format support the definition (down to which register a parameter is passed in, as well as its type) of each parameter.
If the concept is workable, then difficulty of implementing it shouldn't be a barrier.
It is possible for ELF object files by encoding the parameter type information in the version string.  How practical that would be, I dunno; haven't tried it in real life.

In general, I don't see exactly how a compiler could really use anything but the "default" ABI when compiling library code that will be called by other code: it just does not have sufficient information to make a qualified guess.

However, things like knowing which registers a function overwrites at compile time, would let a compiler generate more efficient code: no need to save unused registers.

On a CPU like ATmega328P there are 32 registers, the C compiler (that I used) avoids using some registers because the libraries that may be incorporated use some of them by convention. But if the compiler was able to know which library components are used, and those components said something like"I need 3 registers but don't care which" then the compiler could optimise its use of registers.
This is exactly why I use extended asm (https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html) when compiling C or C++ static inline functions with GCC, Clang, or Intel CC (using adding the __attribute__((always_inline)) function attribute, to force inlining of the function).  Instead of specifying the exact registers, you use numbers (%0, %1) or names (%[foo], %[bar]), and use constraints (https://gcc.gnu.org/onlinedocs/gcc/Constraints.html#Constraints) to specify which kind of register can be used for the inlined function.
That way, the compiler includes the code at each call site, but is free to choose different registers to use, within the specified constraints.

It is also useful to think about whether *statically* compiled code needs to conform to a specific ABI or not.  Say, if the compiler does two passes (across the entire project), with the initial pass basically looking at which registers it needs to use across the entire project, for example generating intermediate (but target-specific) code representation, with the second pass doing register selection and related optimizations and generating the final machine code; it could select and record the calling convention for each function.  That calling convention information would not be needed at runtime at all, but it would be necessary for debugging and in-circuit emulation.

Is it worth it?  I dunno.  Like I keep harping, one really should look at and think about the generated code (or what kind of code one wants it to generate), before deciding.

(Dynamically linked code is different, because anything but a specific ABI agreed upon beforehand would require runtime glue code generation.  I don't like that.)
Title: Re: The Imperium programming language - IPL
Post by: JPortici on January 22, 2023, 08:03:24 am
Then there's another variant of the declare statement using "as":

RE: the As keywork, one programming language i use extensively have recently introduced explicit casting.
Since this language is derived from java, it follows the "everything is an object" rule. To cast one object you add .As(Type) after the object name, then you can use all the methods associated with the type
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 22, 2023, 11:14:16 am
@jportici
Scala?
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 22, 2023, 11:32:10 am
There is at least one language that seems to have many of the facilities the OP believes will be useful. It is contempory with his "favoured" language PL/1.

One of the things that is messy in assembler is using fractions in fixed point arithmetic. Floating point is fine when it is part of the instruction set, but not every microprocessor has FP.

Fixed point was implemented in an HLL in the mid 60s....

There are three types of number, floating-point, fixed-point and integer. Except in certain part-word table-elements (see Section 4.4.2.2, “Part-Word Table Elements” [18]), all three types are signed. Numeric type is indicated by the word FLOATING or INTEGER or by the word FIXED followed by the scaling constants which must be given numerically, e.g.
FIXED(13,5) This specifies five fractional bits and a minimum of 13 bits to represent the number as a whole, including the fractional bits and sign. The number of fractional bits may be negative, zero, or positive, and may cause the binary point to fall outside the significant field of the number.

The same language also has something rather nice for bit manipulations...

Part-Words ... the desired field is selected by a prefix of the form BITS[ Totalbits , Bitposition] in front of the primary to be operated upon. The result of this operation is a positive integer value of width Totalbits and in units of the bit at Bitposition.
BITS[2,6]x := 3 has the effect of placing the binary digits 11 in bits 7 and 6 of the word allocated to x

Packed Data ... and one in which the data format is declared in advance. In the latter system, with which this section is concerned, the format is replicated to form a table. A group of n words is arbitrarily partitioned into bit-fields (with no fields crossing a word boundary), and the same partitioning is applied to as many such groups (m say) as are required. The total data-space for a table is thus nm words. Each group is known as a table-entry. The fields are named, so that a combination of field identifier and entry index selects data from all or part of one computer word, known as a table-element. The elements in a table may occupy overlapping fields, and need not together fill all the available space in the entry.
...
Part-Word Table Elements Elements which occupy fields narrower than a computer word (and only such elements) are declared in forms such as
rain UNSIGNED(4, 2) 2,0;
humidity UNSIGNED(6,6) 2,8;
temperature (10,2) 2,14;

It also has macros, for those that like such abominations...

A compiler embodies a macro processor, which may be regarded as a self-contained routine which processes the text of the program before passing it on to the compiler proper. Its function is to enable the programmer to define and to use convenient macro names, in the form of identifiers, to stand in place of cumbersome or obscure portions of text, typically code statements.

The language's definition is online at http://www.xgc-tek.com/manuals/pdf/xgc-c66-1.7-rm.pdf (http://www.xgc-tek.com/manuals/pdf/xgc-c66-1.7-rm.pdf)
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 22, 2023, 12:01:42 pm
ATmega328P there are 32 registers

MIPS has 32 registers, too, 1 is always zero, 4 are always reserved.

And, when you're completly lost with your ICE session, this has always been the best thing to have, a constant of the universe, like the speed of light, from OS to barebones applications, those registers are there to provide exactly the information you need :D

(you appreciate it even better on SPARC)
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 22, 2023, 12:16:34 pm
I had not seen the term interval arithmetic before although I think I know the concept well. Maybe a tool?

Good for fuzzy-logic, so, good for uc-robotics.
(just, it's 90s old-school, but still good)
Title: Re: The Imperium programming language - IPL
Post by: JPortici on January 22, 2023, 01:21:10 pm
@jportici
Scala?

No, Basic 4 Android :D
A huge project that provides a common language framework to write apps for Android, iOS, PC (java).
Free to use for android/pc, paid updates for iOS but actually really cheap compared to other options.
The biggest plus, to me, is how it handles layouts. It's actually easy and the result is predictable and there is none of that declarative bullshit app programmers seem to be fond of these days (kotlin, flutter, looking at you.)
The fact that the language itself is a dialect of basic also means that it's actually understandable by people. It's not trying to reinvent the wheel by changing names to things, or strange cryptic syntax.
Community is great, libraries provided by the community actually work, and if they are popular they are adopted by the project creator, it's not the clusterfuck of cobbled together libraries like you find in flutter (ble library in flutter is too basic. An user threw up a library to give more functionality, the github has 700+ issues and most of the replies are go to this fork of the library, which has 500+ issues. I am terrified by that environment.)
Tutorial are always up to date, when something big changes in the language, like start with a template or with an empty application, or when a class becomes an internal library, the tutorial is rewritten with both approaches. You can't dream of better support
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 22, 2023, 02:03:49 pm
One of the things that is messy in assembler is using fractions in fixed point arithmetic. Floating point is fine when it is part of the instruction set, but not every microprocessor has FP.

So being able to define something as (in an arbitrary language) binary [8\8] would be nice, an 8 bit integer part and an 8 bit fraction part. Or [0\24], [16\16] etc. The code required is simple, just shift bits or add zero bytes before computation. But interpreting what goes on is not so clear. Being able to express it in the language would be helpful.

Is this already a thing, or is it too obscure to be worried about? Languages I've encountered assume int is integer (funny about that) and if fractions are thing, it is done in floating point with a floating point library if not native to the architecture.

This is already committed to, PL/I supported this out of the box and there are numerous use cases for it in microcontroller applications.

e.g

Code: [Select]

dcl rate fixed bin(15,8);


That's a 16 bit number, one sign bit, the last 8 digits being after decimal point. I've also added quite flexible notation for literals in different bases.

Code: [Select]

rate = 1101.0011101:b;


Note, PL/I had optional abbreviated forms for many keywords and would default many things too. I think it was too tolerant though, so its likely that area needs reviewing. These are all legal in PL/I

Code: [Select]
dcl counter;     // defaults to fixed decimal 5 digits
declare rate float binary(32);
dcl size bin(15); // defaults to fixed

basically there are various optional attributes and if not specified will be defaulted
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 22, 2023, 02:36:34 pm
@jportici
Scala?

No, Basic 4 Android :D
A huge project that provides a common language framework to write apps for Android, iOS, PC (java).
Free to use for android/pc, paid updates for iOS but actually really cheap compared to other options.
The biggest plus, to me, is how it handles layouts. It's actually easy and the result is predictable and there is none of that declarative bullshit app programmers seem to be fond of these days (kotlin, flutter, looking at you.)
The fact that the language itself is a dialect of basic also means that it's actually understandable by people. It's not trying to reinvent the wheel by changing names to things, or strange cryptic syntax.
Community is great, libraries provided by the community actually work, and if they are popular they are adopted by the project creator, it's not the clusterfuck of cobbled together libraries like you find in flutter (ble library in flutter is too basic. An user threw up a library to give more functionality, the github has 700+ issues and most of the replies are go to this fork of the library, which has 500+ issues. I am terrified by that environment.)
Tutorial are always up to date, when something big changes in the language, like start with a template or with an empty application, or when a class becomes an internal library, the tutorial is rewritten with both approaches. You can't dream of better support

Laying out user interfaces declaratively is by far the most logical approach. Stuff like resizing, resolution independence and so on, are much cleaner.

I started using Blazor professionally for new web apps, and Maui is superb for cross platform UI development.

I also used WPF for several years after writing apps with Windows Forms for years, frankly Microsoft are starting own cross platform, ignore them at your peril.

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 22, 2023, 05:29:58 pm
The grammar for structures took some experimentation, partly because this is a new grammar, not PL/I not C or anything.

Anyway after some design effort this is now the syntax for enums and structs

Code: [Select]
      type baud_rates enum bit(8);
           first  = 1010 0011:b;
           second = 1110 1101:b;
           third  = 1101 0111:b;
           fourth = 0010 1100:b;
      end;

      type Outer                 structure;
           cpu_id                bin(31);
           Inner                 structure;
                cpu_mode         bin(31);
                cpu_rev          bin(15);
                large as         Header;
                Flag_table       structure;
                   root          pointer;
                   depth         fixed bin(15);
                end;   
                label            string(64);
           end;
           owner                 string(64);
           speed as              baud_rates;
           counter               string(32);
      end;

This passes various tests and nested structures parse recursively without any problems.
Title: Re: The Imperium programming language - IPL
Post by: SiliconWizard on January 22, 2023, 07:52:50 pm
I had not seen the term interval arithmetic before although I think I know the concept well. Maybe a tool?

Good for fuzzy-logic, so, good for uc-robotics.
(just, it's 90s old-school, but still good)

It's good for many things. :)
And math concepts are never old school.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 03:17:12 pm
Then there's another variant of the declare statement using "as":

RE: the As keywork, one programming language i use extensively have recently introduced explicit casting.
Since this language is derived from java, it follows the "everything is an object" rule. To cast one object you add .As(Type) after the object name, then you can use all the methods associated with the type

Yes, C# too has an 'as' keyword, lets you declare a reference to some object as specifically some class/interface that the object is known to implement rather than the class/interface that the reference was declared to be.



Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 03:19:23 pm
I had not seen the term interval arithmetic before although I think I know the concept well. Maybe a tool?

Good for fuzzy-logic, so, good for uc-robotics.
(just, it's 90s old-school, but still good)

It's good for many things. :)
And math concepts are never old school.

Well, I never eve heard of "interval arithmetic" is it something that has specific support in some programming language? (other than Mathematica etc).
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 03:22:59 pm
There is at least one language that seems to have many of the facilities the OP believes will be useful. It is contempory with his "favoured" language PL/1.

One of the things that is messy in assembler is using fractions in fixed point arithmetic. Floating point is fine when it is part of the instruction set, but not every microprocessor has FP.

Fixed point was implemented in an HLL in the mid 60s....

There are three types of number, floating-point, fixed-point and integer. Except in certain part-word table-elements (see Section 4.4.2.2, “Part-Word Table Elements” [18]), all three types are signed. Numeric type is indicated by the word FLOATING or INTEGER or by the word FIXED followed by the scaling constants which must be given numerically, e.g.
FIXED(13,5) This specifies five fractional bits and a minimum of 13 bits to represent the number as a whole, including the fractional bits and sign. The number of fractional bits may be negative, zero, or positive, and may cause the binary point to fall outside the significant field of the number.

The same language also has something rather nice for bit manipulations...

Part-Words ... the desired field is selected by a prefix of the form BITS[ Totalbits , Bitposition] in front of the primary to be operated upon. The result of this operation is a positive integer value of width Totalbits and in units of the bit at Bitposition.
BITS[2,6]x := 3 has the effect of placing the binary digits 11 in bits 7 and 6 of the word allocated to x

Packed Data ... and one in which the data format is declared in advance. In the latter system, with which this section is concerned, the format is replicated to form a table. A group of n words is arbitrarily partitioned into bit-fields (with no fields crossing a word boundary), and the same partitioning is applied to as many such groups (m say) as are required. The total data-space for a table is thus nm words. Each group is known as a table-entry. The fields are named, so that a combination of field identifier and entry index selects data from all or part of one computer word, known as a table-element. The elements in a table may occupy overlapping fields, and need not together fill all the available space in the entry.
...
Part-Word Table Elements Elements which occupy fields narrower than a computer word (and only such elements) are declared in forms such as
rain UNSIGNED(4, 2) 2,0;
humidity UNSIGNED(6,6) 2,8;
temperature (10,2) 2,14;

It also has macros, for those that like such abominations...

A compiler embodies a macro processor, which may be regarded as a self-contained routine which processes the text of the program before passing it on to the compiler proper. Its function is to enable the programmer to define and to use convenient macro names, in the form of identifiers, to stand in place of cumbersome or obscure portions of text, typically code statements.

The language's definition is online at http://www.xgc-tek.com/manuals/pdf/xgc-c66-1.7-rm.pdf (http://www.xgc-tek.com/manuals/pdf/xgc-c66-1.7-rm.pdf)

This is interesting, I've not looked at Coral at all and I am always interested in older languages, interested to see how certain ideas were implemented, so thanks for mentioning it. Also nice to see language invented in my home country, other than LEO that is.


Title: Re: The Imperium programming language - IPL
Post by: AndyBeez on January 23, 2023, 03:34:06 pm
So from a programmer's perspective, how does this IPL thing handle event/interupt bindings? Is there a notion of asynchronous threading - maybe across DMA boundaries?

In other words, does IPL facilitate a callback or declarative closure pattern that handles events from outside of the main thread/loop/queue?
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 03:46:33 pm
So from a programmer's perspective, how does this IPL thing handle event/interupt bindings? Is there a notion of asynchronous threading - maybe across DMA boundaries?

In other words, does IPL facilitate a callback or declarative closure pattern that handles events from outside of the main thread/loop/queue?

You'd need to elaborate here, not entirely sure what you're asking me. There is no thread/loop/queue insofar as the language itself is defined. The language supports synchronous interrupts in the form of exception support, an exception can be system generated or explicitly generated by code.

Asynchronous interrupts are supported by defining procedures declared as "interrupt" such procedures have specific register save/restore semantics.

This isn't expected to be particularly challenging to implement but perhaps there's scope for innovation here if there are ideas that could be included in this area, for example you mention DMA, wouldn't that just require basic interrupt support?
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 03:52:08 pm
Regarding CORAL 66, there's a British Standards Document (https://standards.globalspec.com/std/477987/BS%205905) - sadly not free!

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 04:02:47 pm
Not sure why, but reading about CORAL 66 takes me back to my childhood! When British innovation was more visible. It brings back memories of watching Dr. Who followed by Tomorrow's World and the Sky at night, fond memories...
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 06:03:20 pm
Here's some actual real life CORAL 66 (http://www.gtoal.com/languages/coral66/restore.cor) source, it's very odd that the language seems to need single quotes around keywords, never seen that before, I wonder why they did that...
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 06:09:17 pm
This reminds me (https://forums.theregister.com/forum/all/2020/05/15/algol_60_at_60/#c_4032874) of serious old days stuff:

Quote
Re: No love for CORAL 66?
It turned out the FORTRAN compiler on the VAX was actually quite clever.

Unfortunately, their CORAL-66 compiler thought it was clever too.

The first professional job I ever did was to help track down a bug in a sonar simulation system, written in CORAL-66 using the MASCOT methodology (defence bods will remember that). It always eventually crashed with a stack overflow, though the amount of run-time before the crash varied considerably.

Turned out that inside a double, nested FOR-loop was an IF statement of the form IF <x> AND <y> THEN... , where x and y were complex expressions.

Of course, if x turns out to be false, you don't have to evaluate y. So the optimising compiler added a jump to the next thing, which happened to be outside of the nested loops. But it left the temporary result for x on the stack. So eventually there would be an overflow.

The number of iterations of the loops and the value of x depended on the input data, but also on how much time the scheduler gave to each of the "concurrent" processes in its simulation of a real-time system. So even if you fed it the same input data, it would run for different times before crashing.

I remember going to a DECUS meeting to tell everyone about this. But it turned out that we were the last to know :(

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 06:56:28 pm
As part of the rationalization of the language, I've replaced "auto" / "automatic" with "stack", were all grown ups, time we stopped using these arcane terms.

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 07:37:47 pm
What do others think about namespace privacy, now that namespaces are part of the language?

I want to allow an optional "private" to a namespace defintion:

Code: [Select]
namespace system.kernel private;

   // Everything in here can only be accessed by code that is also inside the "namespace system.kernel private" namespace.

end;

Code like this is public:

Code: [Select]
namespace system.kernel;

   // Code here is not private

end;

There can be multiple source files that contain the same namespace names of course and a single source file can contain multiple namespaces.

During compilation, all like-named namespaces are coalesced and so will have a public part and an optional private part. During the reference resolution stage accessibility can be checked.

This idea strikes me as better than labelling individual procedures and stuff as private, helping to keep all private code together more easily.

In a sense we can think of every namespace having a possibly empty private area.

It might also be feasible to "layer" the privacy too, I have to think about that...





Title: Re: The Imperium programming language - IPL
Post by: gf on January 23, 2023, 07:45:14 pm
As part of the rationalization of the language, I've replaced "auto" / "automatic" with "stack", were all grown ups, time we stopped using these arcane terms.

Hardly anyone uses 'auto' in C, and in C++11 and later it is a placeholder type specifyer, meaning that the declared variable derives its type from the initializer (e.g. 'auto f = 3.0;' declares variable 'f' with type 'double').
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 07:53:02 pm
As part of the rationalization of the language, I've replaced "auto" / "automatic" with "stack", were all grown ups, time we stopped using these arcane terms.

Hardly anyone uses 'auto' in C, and in C++11 and later it is a placeholder type specifier, meaning that the declared variable derives its type from the initializer (e.g. 'auto f = 3.0;' declares variable 'f' with type 'double').

Good Lord! really? I guess it's so hard to add new keywords to the language that it's now become necessary to give the same keyword different meanings in different contexts!

Title: Re: The Imperium programming language - IPL
Post by: cfbsoftware on January 23, 2023, 10:08:17 pm
Here's some actual real life CORAL 66 (http://www.gtoal.com/languages/coral66/restore.cor) source, it's very odd that the language seems to need single quotes around keywords, never seen that before, I wonder why they did that...
Another way to differentiate between keywords and other identifiers to allow new keywords to be added later without causing problems?

I've seen this in some Algol 60 programs, also discussed here:

http://cs.ecs.baylor.edu/~maurer/SieveE/Algol60.htm (http://cs.ecs.baylor.edu/~maurer/SieveE/Algol60.htm)
Quote
Some compilers forced programmers to put all keywords in quotes, which was permitted by the language definition.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 10:20:19 pm
Here's some actual real life CORAL 66 (http://www.gtoal.com/languages/coral66/restore.cor) source, it's very odd that the language seems to need single quotes around keywords, never seen that before, I wonder why they did that...
Another way to differentiate between keywords and other identifiers to allow new keywords to be added later without causing problems?

I've seen this in some Algol 60 programs, also discussed here:

http://cs.ecs.baylor.edu/~maurer/SieveE/Algol60.htm (http://cs.ecs.baylor.edu/~maurer/SieveE/Algol60.htm)
Quote
Some compilers forced programmers to put all keywords in quotes, which was permitted by the language definition.

Thanks, that was an interesting little page. I do have some faint recollection of the quotes around keywords, very distant memory but where, why and what I have no idea, but I do think I encountered it somewhere once... I suspect you're right, it was a way of allowing new ones without breaking old source code.

Those early languages and compilers deserve a lot of respect, even sitting at a terminal was a rare privilege for most programmers in the 1960s.  The guys who wrote PL/I were kind of forced to write the compiler mostly in assembler. IBM were not interested in stuff that they didn't own, so using Algol to write it was not an option, that kind of closed mindedness is mainly why PL/I never became more popular or more established outside of IBM environments, but that's not the only mistake IBM made as history shows!

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 23, 2023, 11:46:05 pm
Well it's been two months pretty much to the day since this thread was started. A lot has happened, the language has a name "IPL", the grammar is now very solid and stable and arbitrarily complex - legal - source code can be parsed reliably and a parse tree created.

The adaptation from a base PL/I syntax has gone rather well, and the language now supports name spaces, internationalization, type definitions, multiline string literals, enums, free form numeric literals in multiple bases and embedded spaces, new label syntax, Unicode, various shift related operators and more.

The current state of play is summarized here (https://github.com/Steadsoft/imperium/blob/main/README.md) for those interested.

There are several posters here who have been truly helpful and constructive, and I'd like to thank them for their valuable insights and ideas.

I'm stepping back from this work for a time now, but will be actively monitoring the thread, thanks to all.




Title: Re: The Imperium programming language - IPL
Post by: JPortici on January 24, 2023, 06:34:44 am
As part of the rationalization of the language, I've replaced "auto" / "automatic" with "stack", were all grown ups, time we stopped using these arcane terms.

Hardly anyone uses 'auto' in C, and in C++11 and later it is a placeholder type specifier, meaning that the declared variable derives its type from the initializer (e.g. 'auto f = 3.0;' declares variable 'f' with type 'double').

Good Lord! really? I guess it's so hard to add new keywords to the language that it's now become necessary to give the same keyword different meanings in different contexts!

no it's because a local variable that isn't "static" is implicitly "auto", the same way it's redundant to add "extern" to a definition in a header file
Incidentally i was re-reading "Expert C programming" yesterday, there they explain how auto is actually from when they thought a developer would decide between "auto"matic storage (on the stack) or "register" storage (on of course a register)
soon they realized that compilers produced better code if neither would be used.

I guess in C++ they decided to reuse a word that was already reserved, but abandoned
Title: Re: The Imperium programming language - IPL
Post by: westfw on January 25, 2023, 12:10:03 am
Quote
the language has a name "IPL", the grammar is now very solid and stable and arbitrarily complex - legal - source code can be parsed reliably and a parse tree created ...
...
I'm stepping back from this work for a time now...
This is the sort of thing that drove me crazy back in my college "compilers" class.
I learned a hell of a lot about lexical analysis, parsing, and grammars, but the class essentially STOPPED and hand-waved away the actual code-generation (including any optimization steps that might or might not occur.)  At the time I was a heavy-duty assembly-language programmer, and most of what I cared about was whether a compiler could come close to duplicating the efficiency of code I could write in assembly.

So I used to think that C owed its success and longevity to the small footprint of the require "runtime environment."  But now I want to add that "code generation" for C code is "obvious."  You can look at C code and picture what the code generated ought to look like.  Perhaps the compiler optimizes that weirdly, perhaps not.  Maybe this is what people OUGHT to mean when they say "C is just a high level assembler."
As languages get more complex, the code generation gets considerably LESS obvious.
I always figured that the failure of PL/1, even with IBM behind it, was that the compiler ended up being too complicated to port (in a meaningful way) to new architectures.  I don't know if that was run-time environment related, code generation related, compiler size related, or some combination.

So - how is IPL doing in the code generation department?  What's the first target?

Title: Re: The Imperium programming language - IPL
Post by: ataradov on January 25, 2023, 12:48:51 am
This is how 99% of "I want to make my own compiler" threads end up being. Grammar is like 1% of the work and the easiest part at that.

I can sort of understand this for the education though. Code gen past the primitive stack machine with AST traversal and direct generation is really hard work, you would have to dedicate multiple semesters to that, and nobody has time for that.

And a lot of this work is not purely algorithmic as academics like, but more practical. There is no real theories, you just need to sit down and code. And the code gets messy and hard. This is the part where most new languages give up.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 01:18:08 am
Quote
the language has a name "IPL", the grammar is now very solid and stable and arbitrarily complex - legal - source code can be parsed reliably and a parse tree created ...
...
I'm stepping back from this work for a time now...
This is the sort of thing that drove me crazy back in my college "compilers" class.
I learned a hell of a lot about lexical analysis, parsing, and grammars, but the class essentially STOPPED and hand-waved away the actual code-generation (including any optimization steps that might or might not occur.)  At the time I was a heavy-duty assembly-language programmer, and most of what I cared about was whether a compiler could come close to duplicating the efficiency of code I could write in assembly.

So I used to think that C owed its success and longevity to the small footprint of the require "runtime environment."  But now I want to add that "code generation" for C code is "obvious."  You can look at C code and picture what the code generated ought to look like.  Perhaps the compiler optimizes that weirdly, perhaps not.  Maybe this is what people OUGHT to mean when they say "C is just a high level assembler."
As languages get more complex, the code generation gets considerably LESS obvious.
I always figured that the failure of PL/1, even with IBM behind it, was that the compiler ended up being too complicated to port (in a meaningful way) to new architectures.  I don't know if that was run-time environment related, code generation related, compiler size related, or some combination.

So - how is IPL doing in the code generation department?  What's the first target?

Hi,

I want to recommend a great (if a little dated) book on compilers. It was the best of five books I used in the 90s for compiler implementation and explained the code generation far better than any of the other books.

Here's the PDF (https://www.eis.mdx.ac.uk/staffpages/r_bornat/books/compiling.pdf), I have a paper back copy, you can find them around if you prefer that (I do).

The writer understands hardware, assembler and I suspect you'd like the book, it was like a missing piece of the puzzle for me back then, frankly I learned more about compilers from that book than I did from the "Dragon" or the (huge) "Compiler Design in C (https://slebok.github.io/dyol/books/CD-AH.jpg)".

These were all good books but the Bornat book showed the author's passion for the subject, he knew his stuff (it contains one if the very few, earliest discussions of coroutines).

I also tend to agree, the gulf between the front end of a compiler and the raw code bytes that get generated is covered poorly in the literature (when compared to parsing etc) and this was a big struggle for me, Bornat's book was a breath of fresh air frankly.

Yes PL/I faded because of several reasons, one of them was absolutely the complexity of the compiler. Most of the IBM compiler was written in assembler, by very experienced mainframe assembler gurus. It was regarded (in the literature) as the most complex language compiler ever produced.

Even today with the powerful tools we have, the "full IBM PL/I" language is daunting, it included real multitasking and I/O as an integral part of the language.

But it also faded because IBM were IBM, there was zero interest from them in sharing a language or agreeing to standardize it, it was - like much of IBM's products - proprietary, a means to and end, dominate the computer market basically.

For various marketing reasons it was used far more in Europe, particularly Eastern Europe, than it was in the US.

So you're right it was as non-portable as it could get! But it did catch the eye of various software engineers at the time and steps were taken to try and standardize it because many of the concepts it introduced were original and the language had clearly been truly designed, with a list of requirements that were addressed professionally over many months by a host of participants.

But enough of that! As for IPL, the project (if I can call it that) is still in its infancy, the code generation aspects are something I am looking forward too. There's a lot out there that did not exist in the 1990s when I created my own PL/I implementation. It seems to me that LLVM is where the action will be. That offers a powerful abstract CPU that one can generate code for and then, leverage LLVM code generators that can translate that abstract "assembler" to very specific targets.

I know only a little about LLVM, but from what I read that does for the back end what tools like Antlr4 do for the front end. Having created a real working compiler that generated Win32 COFF object and dll files, I'm very relaxed about the code generation, I've done it and seen source code compile, link and run and I'm comfortable with the problem domain.

So I'm not sure if I'm making sense, but I'm not fazed with any aspect of this undertaking, I'm fortunate having done it before and it's a total accident that I'm getting interested in it again.

My focus the past few months has been (to the irritation of some here) "the grammar" and that's because too few programming languages really, truly design the grammar. They seem (and this is true of the vast majority) to adopt some existing language (usually C, Java, Javascript) and then hang on the coat tails.

The problem here though is that a grammar is like clay, once it dries and the shape is formed, it is what it is and there's no going back. What I've been doing the past few months is like a potter at his wheel getting interesting forms but always keeping the clay wet, not letting it set just yet.

I don't want the clay to set until it reaches a point where all the possible innovations and changes have been made, the bad ones cast aside and the good ones nurtured, at some point the grammar will get to the point of "That's it boys, she's ready, lets build her" so to speak!

That point is now close, only this afternoon I reviewed Verilog's syntax and encountered their "reduction" operators. Would a language "need" these? an MCU oriented language? I don't know, but they are neat and if the have a use in FPGA design so that's pretty close to hardware.

Not sure how hardware "oriented" you are but here's a guy I'd have loved to have met (https://en.wikipedia.org/wiki/Bob_Widlar), I like his style.

Later!















Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 01:30:37 am
This is how 99% of "I want to make my own compiler" threads end up being. Grammar is like 1% of the work and the easiest part at that.

With your never ending put downs and defeatism, this comes as no surprise. But please support the bizarre unscientific claim that the grammar design is 1% of the effort in designing a programming language? do you have any evidence? any data?

I can sort of understand this for the education though. Code gen past the primitive stack machine with AST traversal and direct generation is really hard work, you would have to dedicate multiple semesters to that, and nobody has time for that.

Don't assume everybody else has the same shortcomings as you do, I guess you're just not cut out for this subject.

And a lot of this work is not purely algorithmic as academics like, but more practical. There is no real theories, you just need to sit down and code. And the code gets messy and hard. This is the part where most new languages give up.

How many languages have you designed? how many compilers have you built and delivered? Why should anyone listen to your opinions on compilers when you've never written one? Why do you think your opinion should be taken seriously when you apparently struggle so much to even understand the basics of the subject?

If you find it hard to write code, if it gets messy, then take up another vocation, one that doesn't make you so bitter and grumpy.

The more you post here the more foolish you make yourself look, you do realize that? don't you?





Title: Re: The Imperium programming language - IPL
Post by: MIS42N on January 25, 2023, 06:54:11 am
But now I want to add that "code generation" for C code is "obvious."  You can look at C code and picture what the code generated ought to look like.  Perhaps the compiler optimizes that weirdly, perhaps not.  Maybe this is what people OUGHT to mean when they say "C is just a high level assembler."
I agree.

Any messy algorithm I code in VB script then translate it to assembler when the logic is right. I am then just a human compiler/optimiser.

The advantage of a higher level language is it doesn't make mistakes. Testing A < B for a multi byte quantity on an 8-bit CPU requires a few assembler instructions and remembering how to carry forward or get out of the comparison. Easy to accidentally say BCS when should have said BCC. The compiler gets it right every time. I am lazy and just throw in what I guess it to be then run an emulator to sort it out. Once a function is worked out, just cut and paste. Something that gets used more than a couple of times, make a macro. It takes longer to write, but the payoff comes in speed.
Title: Re: The Imperium programming language - IPL
Post by: DC1MC on January 25, 2023, 06:59:49 am
"Parturient montes, nascetur ridiculus mus"  :-DD
Title: Re: The Imperium programming language - IPL
Post by: Siwastaja on January 25, 2023, 07:53:27 am
This is how 99% of "I want to make my own compiler" threads end up being. Grammar is like 1% of the work and the easiest part at that.

With your never ending put downs and defeatism, this comes as no surprise. But please support the bizarre unscientific claim that the grammar design is 1% of the effort in designing a programming language? do you have any evidence? any data?

We will soon* have yet another data point. Now that you have finished your perfect grammar within two months of forum shitposting and ignoring all the experts, and had a bit of experimentation with off-the-shelf lexical tools, maybe a few hours of actual work, let's see how long it takes for you to have actual code generation which works for non-trivial pieces of code. Let's say, something you can compile the compiler itself with. Any single actual CPU architecture, maybe x86, or maybe ARMV7-M or RISCV. You should also have some kind of standard library with basic IO functionality, even for a single OS (POSIX or Windows).

*) or maybe not so soon. I'm not holding my breath

Then we can estimate the percentage that went into the design of grammar.
Title: Re: The Imperium programming language - IPL
Post by: magic on January 25, 2023, 10:03:19 am
For the final project on my compilers course at the uni I wanted to deliver something that I wouldn't be too embarrassed about.

So definitely not a simple stack machine, but real register allocation with SSA, I also recall some rescheduling instruction to reduce the amount of data values alive at a time, constant propagation, elimination of pointless arithmetic (think x+0), dead code, tail call optimizations...

The humble goal was to write simple, recursive implementations of factorial, fibonacci and similar things and have them compiled down to a few lines of clean assembly.

This took me three weeks, several hours per day. My conclusion was that compilers can be done, but they are not exactly trivial :D

edit
That being said, it is my understanding that nowadays one can save a lot of time by writing a frontend for LLVM/GCC. These are supposed to have a lot of the hard stuff and all the low level code generation handled for you. Although when it comes to portability, I seem to recall that some of their CPU abstractions are still somewhat leaky...
Title: Re: The Imperium programming language - IPL
Post by: Nominal Animal on January 25, 2023, 10:09:32 am
My interpretation of the essential content of this thread:

"I have now done the hard but brilliant work of defining this new, superior programming language.
As my own time is better spent elsewhere, I shall leave the straightforward task of implementation to the lesser, more mechanically oriented minds, those better suited for such tasks."

I'm sure everyone working in software has met the type: "Here's my million dollar idea: make a new Facebook, but cheaper!  If you implement it, I'll cut you in on 5% of the profits.  Think of the visibility you'll get!"
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 10:16:23 am
I want to recommend a great (if a little dated) book on compilers. It was the best of five books I used in the 90s for compiler implementation and explained the code generation far better than any of the other books.
...
These were all good books but the Bornat book showed the author's passion for the subject, he knew his stuff (it contains one if the very few, earliest discussions of coroutines).

Based on a quick skim, that book is very dated - first written in 1979 with examples in BCPL. There are, apparently some updates, but I don't know what they are.

I'm sure that it represents a good exposition of part of the state of the art back then, but now it is woefully inadequate.

A few observations:

So, while it might be a reasonable exposition of compiler front ends, it doesn't consider what's happening at runtime in modern MCUs or processors.

Quote
My focus the past few months has been (to the irritation of some here) "the grammar" and that's because too few programming languages really, truly design the grammar. They seem (and this is true of the vast majority) to adopt some existing language (usually C, Java, Javascript) and then hang on the coat tails.

The problem here though is that a grammar is like clay, once it dries and the shape is formed, it is what it is and there's no going back. What I've been doing the past few months is like a potter at his wheel getting interesting forms but always keeping the clay wet, not letting it set just yet.

I agree.

But I'm not sure you understand how much the compiler backend and runtime behaviour on modern hardware will affect the language concepts.
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 25, 2023, 11:46:49 am
Reading the readme, op wants to discuss, but none of the points we have commented (e.g. Babylon is stupid idea, rot@what is stupid idea, etc) are there in the readme.

So what's the point here? The op daily requests attention for nothing but his "grammar",  and with aggressive arrogance it's nothing but all "write mode".
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 25, 2023, 11:57:38 am
This is how 99% of "I want to make my own compiler" threads end up being. Grammar is like 1% of the work and the easiest part at that.

I can sort of understand this for the education though. Code gen past the primitive stack machine with AST traversal and direct generation is really hard work, you would have to dedicate multiple semesters to that, and nobody has time for that.

And a lot of this work is not purely algorithmic as academics like, but more practical. There is no real theories, you just need to sit down and code. And the code gets messy and hard. This is the part where most new languages give up.

I can confirm this  :o :o :o

my-c experience is exactly like this: one man project, 8 years (6 hours per week ) to have a *decent* machine layer; while, according to my task-planner, adapting the c89 grammar only took 78 hours.


Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 25, 2023, 12:16:03 pm
Quote
My focus the past few months has been (to the irritation of some here) "the grammar" and that's because too few programming languages really, truly design the grammar. They seem (and this is true of the vast majority) to adopt some existing language (usually C, Java, Javascript) and then hang on the coat tails.

The problem here though is that a grammar is like clay, once it dries and the shape is formed, it is what it is and there's no going back. What I've been doing the past few months is like a potter at his wheel getting interesting forms but always keeping the clay wet, not letting it set just yet.

I agree.

... the OP speaks as if he were a luminary university professor with several qualifications with compilers and hardware, worse still, to *justify* the aggressive arrogance he thinks he's right when he wastes people's time with fake endless arguments where we spend the time making comments that HE always ignores.

I agree to one point.

Lot Pop Corn :popcorn:

Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 25, 2023, 01:10:20 pm
@magic
if you want to laugh I can try to find where I parked my old "L interpreter" and post its C code somewhere here.

It's a "complete" toy, well... just ... I emmm emmm "forgot" to implement function calls because I thought the project would take less time, and at some point I realized it would miss the deadline assigned by my teacher for the project.

Teacher: oh, well, it looks like Pascal
Me: yup (  with this face -> 8) ), I perfected my grammar a bit...
Teacher: hey wait… I see examples, BUT no function calls!? !? !?
Me: ... ups, I think *the polish* took me so long that I think I forgot to implement ...  Next commit.
Teacher: sure, next semester!
Me ... umm?
Teacher: I can give you 24/30 now, but you can always re-take the course and re-commit next semester.



lesson learned: the polish is nice to show off but it's not a priority :o :o :o


Title: Re: The Imperium programming language - IPL
Post by: magic on January 25, 2023, 01:31:04 pm
My compiler also had a curious property: compilation time increased exponentially with the number of nested loops, an unintended consequence of my laziness implementing some optimization :palm:

Thankfully, the teacher had no tests for it and I only realized my mistake later :phew:
Title: Re: The Imperium programming language - IPL
Post by: Siwastaja on January 25, 2023, 02:00:12 pm
And finally, only time will test the true usability of a programming language. If no one ends up using it, there probably is something wrong with it. Simple languages which have simple workarounds to problems that always and inevitably occur even to most brilliantly designed languages, have advantage over complex languages which require complex workarounds.
Title: Re: The Imperium programming language - IPL
Post by: DC1MC on January 25, 2023, 02:06:03 pm
"A language that doesn't have everything is actually easier to program in than some that do." -- Dennis M. Ritchie
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 02:20:48 pm
For the final project on my compilers course at the uni I wanted to deliver something that I wouldn't be too embarrassed about.

So definitely not a simple stack machine, but real register allocation with SSA, I also recall some rescheduling instruction to reduce the amount of data values alive at a time, constant propagation, elimination of pointless arithmetic (think x+0), dead code, tail call optimizations...

The humble goal was to write simple, recursive implementations of factorial, fibonacci and similar things and have them compiled down to a few lines of clean assembly.

This took me three weeks, several hours per day. My conclusion was that compilers can be done, but they are not exactly trivial :D

edit
That being said, it is my understanding that nowadays one can save a lot of time by writing a frontend for LLVM/GCC. These are supposed to have a lot of the hard stuff and all the low level code generation handled for you. Although when it comes to portability, I seem to recall that some of their CPU abstractions are still somewhat leaky...

I know little yet about LLVM, I believe it's certainly a sensible way to go though, over a hand crafted code generator. Despite the incessant prattling from the usual self appointed compiler Gestapo, I've shown some of them evidence of my experience, I mentioned this in my first post.

I've designed and written a Pentium code generator for the PL/I language, I've revealed the source code. That was a lot of work, it includes an abstract code generator, an instruction analyzer and opcode generator, a peep hole optimizer, a COFF object file output generator.

So yes that was a lot of work, LLVM promises to reduce that but it will (like Antlr) require that those tool be fully understood and that takes time.

I don't think anybody here has ever designed a grammar yet the prattlers here glibly dismiss its significance. This is likely because the beginner books they've likely read never talk about it, they almost always use an existing language as their grammar.

Grammars need to be designed, until it is, writing code that assumes a grammar is a waste of time because until the grammar is deemed stable, changes to it will be needed, rework to reams of code needed, so why do it? I think many inexperienced developers are like this, the never design, they just "code", I guess its the way they were taught.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 02:53:08 pm
I don't think anybody here has ever designed a grammar yet the prattlers here glibly dismiss its significance.

Probably, and probably not. The latter because it is a solved, and more importantly because other issues are far more problematic. (Examples: memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation)

Quote
This is likely because the beginner books they've likely read never talk about it, they almost always use an existing language as their grammar.

Possibly.

If in order to complete a task you have to do either A or B, then start with the easier.
If in order to complete a task you have to do either A and B, then start with the more difficult.

You have to define the concepts/behaviour and grammar, and do the compilation and optimisation and code generation. You are starting with the the easiest. Shame.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 03:13:24 pm
I don't think anybody here has ever designed a grammar yet the prattlers here glibly dismiss its significance.

Probably, and probably not. The latter because it is a solved, and more importantly because other issues are far more problematic. (Examples: memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation)

So that's not a yes then is it? have you ever designed a programming language? have you ever designed anything? I'm assuming the answer is "No".

It isn't solved, a design isn't done until the goals of the design are met, until the objectives defined for the grammar met, until then it is not "done", do you really not understand what design is?

One of the reasons for this thread existing was simply to gather input in order to define those requirements and goals, but rather than participate you prefer to complain about "progress" and "code generation" things that cannot be progressed efficiently when the goals are still being defined.

Whether some phase of a project is or is not the largest source of challenges, is not what you were discussing, it is the order of the phases you've been lecturing me on.

Building a house generates many more problems than laying the foundations and laying the drainage system, yet if you were building a house you'd do the foundations after building the house I suppose?

What would happen if you had constructed the first twenty floors of a large skyscraper when you discover you didn't design the foundations sufficiently well? let me tell you - you'd be fired as the project manager and replaced by someone who know what "design" means.


Quote
This is likely because the beginner books they've likely read never talk about it, they almost always use an existing language as their grammar.

Possibly.

If in order to complete a task you have to do either A or B, then start with the easier.
If in order to complete a task you have to do either A and B, then start with the more difficult.

This is proof positive that you're a troll now. The order of phases in a project is not dictated by their relative complexity, one cannot build a rocket and send men to the moon without doing other stuff first like deciding on a route, deciding the likely fuel consumption, safety of the crew, costs, support staffing, team roles etc etc.

You have to define the concepts/behaviour and grammar, and do the compilation and optimisation and code generation. You are starting with the the easiest. Shame.

Your trolling is nothing to do with language design at all, it really about your naive "understanding" of project planning and scheduling, you think you are an expert yet your own buffoonery above shows us otherwise.

There are a handful of people in this thread who know how to discuss this subject like adults, you are clearly not one of them.

I happen to have decades of experience as a software project manager, I've earned very large bonuses for delivering multi million dollar projects ahead of schedule, managing risk, maintaining progress, you have nothing to teach me about project management or programming language design.

Like so many of the blowhards here you think designing a programming language can be reduced to "coding" and if one isn't "coding" then they are fools, doomed to failure.

Coding, coding coding..

I know all about coding, here's the hand coded 8,500 lines recursive descent parser (https://github.com/Steadsoft/PLI-2000/blob/main/Compiler%20Source/PASS1.C) for PL/I subset G written in C on a DOS PC with no internet resources.

Or how would like to see the 4,500 line semantic analyzer (https://github.com/Steadsoft/PLI-2000/blob/main/Compiler%20Source/PASS2.C)? what? it's irrelevant? really? Oh well there it is anyway.

Perhaps the symbol table manager and reference resolver (https://github.com/Steadsoft/PLI-2000/blob/main/Compiler%20Source/DECLARE.C) is of some interest? here it is then sorry it's only 750 lines, I'll try harder next time!

Surely the storage allocator (https://github.com/Steadsoft/PLI-2000/blob/main/Compiler%20Source/ALLOCATE.C) is relevant? no? that too is a waste of time? oh well, for the record here it is, again it's only 700 lines, I am sorry about that.

Now your absolute favorite part at last! yes the code generator ! whoopee ! here's the first part, the abstract Pentium code generator (https://github.com/Steadsoft/PLI-2000/blob/main/Compiler%20Source/CODEGEN.C) this is 3,000 lines.

And finally the real deal, the stuff that separate the "men from the boys" as it were, the actual instruction opcode generator (https://github.com/Steadsoft/PLI-2000/blob/main/Compiler%20Source/EMITTER.C), generates real binary instruction stream, another 3,000 lines.

Last but by no means least, the COFF output file generator itself, that's right, yes, it's true, code generators aren't enough, we really need to create an output file with a strict structure and format that a linker can understand, so here it is (https://github.com/Steadsoft/PLI-2000/blob/main/Compiler%20Source/Cofflib.c).

Ooh, I almost forgot, for all those lovers of assembler here, this is the 1,200 line 80386 runtime (https://github.com/Steadsoft/PLI-2000/blob/main/RUNTIME.ASM) (for MS-DOS) that got linked in to every build of a PL/I app.

To cap it all the proof of the pudding as they say! This code ran on DOS (https://github.com/Steadsoft/PLI-2000/blob/main/pl1/PIXELS.PL1).

Now please, pretty please with a cherry on top, stop trolling and let the adults here talk in peace.




Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 03:20:48 pm
This is how 99% of "I want to make my own compiler" threads end up being. Grammar is like 1% of the work and the easiest part at that.

With your never ending put downs and defeatism, this comes as no surprise. But please support the bizarre unscientific claim that the grammar design is 1% of the effort in designing a programming language? do you have any evidence? any data?

We will soon* have yet another data point. Now that you have finished your perfect grammar within two months of forum shitposting and ignoring all the experts, and had a bit of experimentation with off-the-shelf lexical tools, maybe a few hours of actual work, let's see how long it takes for you to have actual code generation which works for non-trivial pieces of code. Let's say, something you can compile the compiler itself with. Any single actual CPU architecture, maybe x86, or maybe ARMV7-M or RISCV. You should also have some kind of standard library with basic IO functionality, even for a single OS (POSIX or Windows).

*) or maybe not so soon. I'm not holding my breath

Then we can estimate the percentage that went into the design of grammar.

Which "expert" have I ignored?

Can you show that grammar design is 1% of the effort needed to design a new language? that's what I asked about, I'd like to see the facts, the evidence supporting that claim, is there any?

Why do you care how long various phases of such a project are? If the code generation takes a week or a year, what of it? why is that important to you?

Your posts are vacuities, utter wastes of time, conversation between us can serve no purpose, goodbye.

Title: Re: The Imperium programming language - IPL
Post by: newbrain on January 25, 2023, 03:42:18 pm
other issues are far more problematic. (Examples: memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation)
One thousand times this.
It took many years for C to have a defined sequencing and memory model, and to tackle aliasing in an optimisation friendly way (I'm not expert enough to say whether they did it 100% correctly).

Hints to the OP on these topic have been mostly ignored. I'm not interested in grammar that much (though I of course have my personal taste).

BTW, my master thesis was, in fact, a compiler for occam, so the grammar was a given - not an issue*.
Being on MS-DOS, the major problem was rather the implementation of concurrency (PAR construct) and channels.
I went for the simplest solution: stack based VM, only minor optimisations, tick based multitasking; I wrote it in Turbo Pascal, but 80% of the runtime library was in assembler.
The generated code was also able to run on a NETBIOS network**, by just adding a configuration file matching channels and procedure with a named machine.
I had the pleasure of it being used in several technical schools to teach some programming concepts.

*And having being exposed to occam, Python's 'indentation is scope' just does not strike me as odd.
**That was the first time I encountered C as the bindings and library for NETBIOS networking available to me were C only. Did not think much of it at the time, but that was before C89 was finalized.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 04:09:07 pm
other issues are far more problematic. (Examples: memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation)
It took many years for C to have a defined sequencing and memory model, and to tackle aliasing in an optimization friendly way (I'm not expert enough to say whether they did it 100% correctly).

Hints to the OP on these topic have been mostly ignored. I'm not interested in grammar that much (though I of course have my personal taste).

You speak of "hints", instead of hinting, state your view, be specific, what exactly do you want to suggest to me? I've not intentionally ignored suggestions that are on-topic, if it seems I have, then I apologize.

So either quote yourself so I can see these "hints" again or state your case, I'm truly interested in things that a language can incorporate that further the goals stated at the outset of this thread.

I liken the grammar to a frame, or substrate, perhaps a clay foundation that one might later paint and embellish.

I am keeping the clay wet just now, it is still subject to change, perhaps small perhaps large, who knows, but until the goals are met that clay must remain plastic, if it sets too soon and we discover a shortcoming too late, after hundreds of hours of painting, then what a shame that would be.

Grammar's are designed, created, they are the bedrock of the language's expressiveness, they help embed human thought into our source code, a poor grammar (like C) can have long lasting negative consequences. Most compiler books speak very little of new grammar design, only about existing grammar representation in say Antlr or Lexx or Yacc and so on, they almost always start using an existing grammar for their discourse.

Few books speak of why the grammar is as it is, the different ways of expressing things and so on.

For example does the IPL grammar I have now lend itself to later support of lambda functions? How can we express anonymous functions in the language? might we end up adjusting the current syntax of function and procedure definitions? these and other questions are all part of this process of grammar design, PL/I never supported anonymous functions, so this is a real question, this is the kind of thing I've been doing with this stage of the language, I'm in no hurry.

(Python's indentation scope is a problem, lose the layout, the spacing and you lose critically important syntactic information, this was a bad grammar design decision IMHO and I'm far from alone (https://www.linkedin.com/pulse/why-python-indentation-sucks-what-can-done-spoiler-alert-mayank-verma/) here).



Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 04:28:27 pm
I don't think anybody here has ever designed a grammar yet the prattlers here glibly dismiss its significance.

Probably, and probably not. The latter because it is a solved, and more importantly because other issues are far more problematic. (Examples: memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation)

So that's not a yes then is it? have you ever designed a programming language? have you ever designed anything? I'm assuming the answer is "No".

Many, many times.

Sometimes my employer has insisted I file patents on what I've invented.

Quote
It isn't solved, a design isn't done until the goals of the design are met, until the objectives defined for the grammar met, until then it is not "done", do you really not understand what design is?

Traditionally difficult part of any greenfield technical project is working out what are worthwhile goals and non-goals. I have decades of experience doing just that.

It is far easier if someone tells you what you have to implement, and leaves you to figure out costs and timescales etc.

Quote
If in order to complete a task you have to do either A or B, then start with the easier.
If in order to complete a task you have to do either A and B, then start with the more difficult.

This is proof positive that you're a troll now. The order of phases in a project is not dictated by their relative complexity, one cannot build a rocket and send men to the moon without doing other stuff first like deciding on a route, deciding the likely fuel consumption, safety of the crew, costs, support staffing, team roles etc etc.

You miss the point, probably in your haste to have a rant.

Analogies are always dangerous, but to follow yours: if you can't build a sufficiently powerful rocket engine, then it is a waste of time/effort to choose a route and recruit astronauts.

Quote
You have to define the concepts/behaviour and grammar, and do the compilation and optimisation and code generation. You are starting with the the easiest. Shame.

Your trolling is nothing to do with language design at all, it really about your naive "understanding" of project planning and scheduling, you think you are an expert yet your own buffoonery above shows us otherwise.

There are a handful of people in this thread who know how to discuss this subject like adults, you are clearly not one of them.

I happen to have decades of experience as a software project manager, I've earned very large bonuses for delivering multi million dollar projects ahead of schedule, managing risk, maintaining progress, you have nothing to teach me about project management or programming language design.

I sampled project management (successfully), but decided it was boring - whereas technical research was interesting.

Otherwise I think your comments reveal more about you and your background than they do about me! I suspect quite a few other participants in this thread do too.

<snipped irrelevant rant>

Quote
Now please, pretty please with a cherry on top, stop trolling and let the adults here talk in peace.

Physician, heal thyself.
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 25, 2023, 04:33:47 pm
Now please, pretty please with a cherry on top, stop trolling and let the adults here talk in peace.

Whatever, again I see nothing but an insanely need of being constantly babysit-ted on the forum  :-//

Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 04:35:03 pm
other issues are far more problematic. (Examples: memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation)
It took many years for C to have a defined sequencing and memory model, and to tackle aliasing in an optimization friendly way (I'm not expert enough to say whether they did it 100% correctly).

Hints to the OP on these topic have been mostly ignored. I'm not interested in grammar that much (though I of course have my personal taste).

You speak of "hints", instead of hinting, state your view, be specific, what exactly do you want to suggest to me? I've not intentionally ignored suggestions that are on-topic, if it seems I have, then I apologize.

Apology accepted.

Start with memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation, multithread parallelism, multicore operation, real-time guarantees... There are others scattered through the thread by several authors; try reading them.
Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 25, 2023, 04:42:57 pm
Quote
I've not intentionally ignored suggestions that are on-topic

 :-DD :-DD :-DD :-DD :-DD

which is even more funny considering how TOXIC and AGGRESSIVE that "not intentionally" goes.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 04:43:37 pm
other issues are far more problematic. (Examples: memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation)
One thousand times this.
It took many years for C to have a defined sequencing and memory model, and to tackle aliasing in an optimisation friendly way (I'm not expert enough to say whether they did it 100% correctly).

Hints to the OP on these topic have been mostly ignored. I'm not interested in grammar that much (though I of course have my personal taste).

BTW, my master thesis was, in fact, a compiler for occam, so the grammar was a given - not an issue*.
Being on MS-DOS, the major problem was rather the implementation of concurrency (PAR construct) and channels.
I went for the simplest solution: stack based VM, only minor optimisations, tick based multitasking; I wrote it in Turbo Pascal, but 80% of the runtime library was in assembler.
The generated code was also able to run on a NETBIOS network**, by just adding a configuration file matching channels and procedure with a named machine.
I had the pleasure of it being used in several technical schools to teach some programming concepts.

*And having being exposed to occam, Python's 'indentation is scope' just does not strike me as odd.
**That was the first time I encountered C as the bindings and library for NETBIOS networking available to me were C only. Did not think much of it at the time, but that was before C89 was finalized.

Sounds fun.

The Occam folding editor looked neat, but I never used it in anger. Its few benefits became moot on hardcopy and when you could get more than 80*25 chars on your display.

Personally I hate and distrust whitespace being used as semantically important. That started with makefiles, where tab-vs-space is significant. The obvious problem arises when someone cuts-and-pastes code: to what level should/shouldn't leading whitespace be automatically inserted/deleted.

The "whitespace denoting scope" has all the hallmarks of neophytes that complain that they have to type too many characters. As someone who wrote their undergraduate project on an ASR33, I've little sympathy with that! For youngsters that haven't had that pleasure, start by realising that "touch typing" is an invalid concept on a teletype :)
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 04:53:34 pm
other issues are far more problematic. (Examples: memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation)
It took many years for C to have a defined sequencing and memory model, and to tackle aliasing in an optimization friendly way (I'm not expert enough to say whether they did it 100% correctly).

Hints to the OP on these topic have been mostly ignored. I'm not interested in grammar that much (though I of course have my personal taste).

You speak of "hints", instead of hinting, state your view, be specific, what exactly do you want to suggest to me? I've not intentionally ignored suggestions that are on-topic, if it seems I have, then I apologize.

Apology accepted.

Start with memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation, multithread parallelism, multicore operation, real-time guarantees... There are others scattered through the thread by several authors; try reading them.

Yes, you did only mention these, what of them? are you insinuating that something specific be incorporated? because there's nothing specific in what you wrote.

These are just headings for God's sake, no substance, what do you mean "memory models" so what EXACTLY would you like to see in a language for this? If these kind of vague subject headings is really all you have then there's not a lot I can do in any practical sense is there? Shall I say "OK I'll include a memory model" or "OK I'll make sure there's an optimizer"? I mean really? would that satisfy you?

You're trolling, pretending to contribute because you can't actually contribute, just endless posting of inane vacuities, coupled with an exaggerated sense of self importance, so sorry buddy I will be ignoring you from now on.

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 05:38:30 pm
I'm leaving this thread due to excessive trolling and spamming, should anyone want to contribute suggestions or questions about this work please do so using Github issues/discussions (https://github.com/Steadsoft/imperium) which I'll respond to regularly.

Regards.

Title: Re: The Imperium programming language - IPL
Post by: JPortici on January 25, 2023, 07:15:04 pm
 :-DD
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 07:35:07 pm
other issues are far more problematic. (Examples: memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation)
It took many years for C to have a defined sequencing and memory model, and to tackle aliasing in an optimization friendly way (I'm not expert enough to say whether they did it 100% correctly).

Hints to the OP on these topic have been mostly ignored. I'm not interested in grammar that much (though I of course have my personal taste).

You speak of "hints", instead of hinting, state your view, be specific, what exactly do you want to suggest to me? I've not intentionally ignored suggestions that are on-topic, if it seems I have, then I apologize.

Apology accepted.

Start with memory models, possibility of aliasing, fundamental concepts embodied in the language, optimisation, multithread parallelism, multicore operation, real-time guarantees... There are others scattered through the thread by several authors; try reading them.

Yes, you did only mention these, what of them? are you insinuating that something specific be incorporated? because there's nothing specific in what you wrote.

It is your choice what to incorporate in your language. But you do need to choose what, why and how.

Quote
These are just headings for God's sake, no substance, what do you mean "memory models" so what EXACTLY would you like to see in a language for this? If these kind of vague subject headings is really all you have then there's not a lot I can do in any practical sense is there? Shall I say "OK I'll include a memory model" or "OK I'll make sure there's an optimizer"? I mean really? would that satisfy you?

Of course they are headings; I'm not designing your language!

I suspect you don't understand why a memory model is necessary, let alone what is involved. Hint: Java had one from the beginning, but had to replace it when the new memory systems became available. Even experts find it difficult. C didn't have one until recently, after Hans Boehm forced people to understand what couldn't be done without one. Even experts didn't understand that!

As for optimisers, are you aware that language features enable/preclude different types of optimisation? Simple example: are you going to allow pointer aliasing (Java, C, C++) or forbid it (xC, Rust)?

It doesn't matter even a tiny bit whether pointer are declared as, say, "int *p" or "REF REF INT p". If you allow pointer aliasing then you preclude many useful optimisations. Grammar is boring; behaviour is interesting.

Quote
You're trolling, pretending to contribute because you can't actually contribute, just endless posting of inane vacuities, coupled with an exaggerated sense of self importance, so sorry buddy I will be ignoring you from now on.

I think the emoji used by JPortici and DiTBho sum up the appropriate response.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 08:23:54 pm
I suspect you don't understand why a memory model is necessary, let alone what is involved. Hint: Java had one from the beginning, but had to replace it when the new memory systems became available. Even experts find it difficult. C didn't have one until recently, after Hans Boehm forced people to understand what couldn't be done without one. Even experts didn't understand that!

I turn my back for five minutes and the barrage of BS just continues unabated, how bored you must be to perpetuate the trolling.

If I'd asked for some ideas for a new electric car perhaps, you'd be the guy who'd "contribute" by posting remarks like "what about wheels then?" or "you've said nothing about headlights" or "nobody cares about easy parts replacement" and other inane vacuous, pointless one liners.

You never bothered to define "memory model", this is why I said your posts here are pretense, pretending to be contributing. You don't define it because you can't, you'll ignore me and wait for someone else to answer for you.

You say "C didn't have one until recently" so when was that then? when did it "get" one? what impact does this have? how did it affect the way you write C code?

What kinds of "memory models" are there? does the C standard use the term "memory model"? have you even looked?

Saying "a memory model is necessary" is utterly meaningless, like saying "drivability is necessary" or "you must have a seats policy", more pretending to know something about this subject when all you really have is emperor's new clothes, fake opinions.

The fact is you can't contribute or else you would have, some here have truly introduced important suggestions and thought provoking ideas but not you because you can't, you want everyone to think you can but you can't, you offer hot air not ideas, fake talk and no walk.

So tell me, what is a "memory model" for, what purpose does it serve? what would happen without one? how many types are there? lots of questions there for you, lets see if you can answer any.
Title: Re: The Imperium programming language - IPL
Post by: gf on January 25, 2023, 08:32:29 pm
So tell me, what is a "memory model" for, what purpose does it serve? what would happen without one? how many types are there? lots of questions there for you, lets see if you can answer any.

https://en.wikipedia.org/wiki/Memory_model_(programming) (https://en.wikipedia.org/wiki/Memory_model_(programming))
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 08:35:28 pm
So tell me, what is a "memory model" for, what purpose does it serve? what would happen without one? how many types are there? lots of questions there for you, lets see if you can answer any.

[url=https://en.wikipedia.org/wiki/Memory_model_(programming)]https://en.wikipedia.org/wiki/Memory_model_(programming) (https://en.wikipedia.org/wiki/Memory_model_(programming))[/url]

No surely you mean:

http://www.cs.cornell.edu/courses/cs2022/2011sp/lectures/lect06.pdf (http://www.cs.cornell.edu/courses/cs2022/2011sp/lectures/lect06.pdf)

or perhaps

https://livebook.manning.com/book/modern-c/chapter-12/ (https://livebook.manning.com/book/modern-c/chapter-12/)

or perhaps

https://www.cs.cmu.edu/~15122/handouts/20-cmem.pdf (https://www.cs.cmu.edu/~15122/handouts/20-cmem.pdf)

or even

https://people.cs.pitt.edu/~xianeizhang/notes/cpp11_mem.html (https://people.cs.pitt.edu/~xianeizhang/notes/cpp11_mem.html)

But that latter one is C++ not C...

Please show me what the C language standard actually says about a "memory model" after all it now "has" one, it got it "recently".

Oh and do note that last document says "A memory model, a.k.a memory consistency model, is a specification", could someone kindly show me this specification for C so I know what people are referring to?



Title: Re: The Imperium programming language - IPL
Post by: gf on January 25, 2023, 08:55:29 pm
So tell me, what is a "memory model" for, what purpose does it serve? what would happen without one? how many types are there? lots of questions there for you, lets see if you can answer any.

[url=https://en.wikipedia.org/wiki/Memory_model_(programming)]https://en.wikipedia.org/wiki/Memory_model_(programming) (https://en.wikipedia.org/wiki/Memory_model_(programming))[/url]

No surely you mean:

http://www.cs.cornell.edu/courses/cs2022/2011sp/lectures/lect06.pdf (http://www.cs.cornell.edu/courses/cs2022/2011sp/lectures/lect06.pdf)

or perhaps

https://livebook.manning.com/book/modern-c/chapter-12/ (https://livebook.manning.com/book/modern-c/chapter-12/)

or perhaps

https://www.cs.cmu.edu/~15122/handouts/20-cmem.pdf (https://www.cs.cmu.edu/~15122/handouts/20-cmem.pdf)

or even

https://people.cs.pitt.edu/~xianeizhang/notes/cpp11_mem.html (https://people.cs.pitt.edu/~xianeizhang/notes/cpp11_mem.html)

But that latter one is C++ not C...

Please show me what the C language standard actually says about a "memory model" after all it now "has" one, it got it "recently".

I think everybody here means the ordering of memory fetch and store operations, as described in the Wikipedia article and in your last link. Yes, your last link is for C++, but for C the semantics are similar. For C, it was introduced in C11. See Wikipedia article.

Quote from: Wikipedia article
The Java Memory Model was the first attempt to provide a comprehensive threading memory model for a popular programming language.[2] After it was established that threads could not be implemented safely as a library without placing certain restrictions on the implementation and, in particular, that the C and C++ standards (C99 and C++03) lacked necessary restrictions,[3][4] the C++ threading subcommittee set to work on suitable memory model; in 2005, they submitted C working document n1131[5] to get the C Committee on board with their efforts. The final revision of the proposed memory model, C++ n2429,[6] was accepted into the C++ draft standard at the October 2007 meeting in Kona.[7] The memory model was then included in the next C++ and C standards, C++11 and C11.[8][9]
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 09:31:11 pm
So tell me, what is a "memory model" for, what purpose does it serve? what would happen without one? how many types are there? lots of questions there for you, lets see if you can answer any.

[url=https://en.wikipedia.org/wiki/Memory_model_(programming)]https://en.wikipedia.org/wiki/Memory_model_(programming) (https://en.wikipedia.org/wiki/Memory_model_(programming))[/url]

No surely you mean:

http://www.cs.cornell.edu/courses/cs2022/2011sp/lectures/lect06.pdf (http://www.cs.cornell.edu/courses/cs2022/2011sp/lectures/lect06.pdf)

or perhaps

https://livebook.manning.com/book/modern-c/chapter-12/ (https://livebook.manning.com/book/modern-c/chapter-12/)

or perhaps

https://www.cs.cmu.edu/~15122/handouts/20-cmem.pdf (https://www.cs.cmu.edu/~15122/handouts/20-cmem.pdf)

or even

https://people.cs.pitt.edu/~xianeizhang/notes/cpp11_mem.html (https://people.cs.pitt.edu/~xianeizhang/notes/cpp11_mem.html)

But that latter one is C++ not C...

Please show me what the C language standard actually says about a "memory model" after all it now "has" one, it got it "recently".

I think everybody here means the ordering of memory fetch and store operations, as described in the Wikipedia article and in your last link. Yes, your last link is for C++, but for C the semantics are similar. For C, it was introduced in C11. See Wikipedia article.

Quote from: Wikipedia article
The Java Memory Model was the first attempt to provide a comprehensive threading memory model for a popular programming language.[2] After it was established that threads could not be implemented safely as a library without placing certain restrictions on the implementation and, in particular, that the C and C++ standards (C99 and C++03) lacked necessary restrictions,[3][4] the C++ threading subcommittee set to work on suitable memory model; in 2005, they submitted C working document n1131[5] to get the C Committee on board with their efforts. The final revision of the proposed memory model, C++ n2429,[6] was accepted into the C++ draft standard at the October 2007 meeting in Kona.[7] The memory model was then included in the next C++ and C standards, C++11 and C11.[8][9]

Right but there are umpteen lectures and articles out there about C and "memory model" that have nothing to do with multithreading semantics.

Furthermore the question of how code generated by the new language, will behave in a multithreaded setting was raised early on and I responded by saying that such a discussion was premature, I acknowledged its relevance but defining that was was for a later phase, it has no bearing on the initial phase which has (for reasons I've explained) concerned with the front end of the language.

This is not ignoring the issue, it is deferring it to some later time, berating me for not discussing defining things now when I already deferred them is just more of the same old incessant attempt to discredit the entire effort.

The code generation and optimization discussions are fine and highly relevant but all in good time, by all means we can cite them and note them for later discussion.

The fact is only a very few want to actively, constructively discuss, the majority here are hell bent on discrediting me at every turn.

I get that some don't want to discuss parsing, grammars, preprocessing and so on. But in that case they can refrain from comment, instead they repeatedly attack because I'm not talking about what they want to talk about.

By talking about front end issues they turn that into accusations that that's all I want to talk about which is a lie, I want to talk about things in the order I outlined.











Title: Re: The Imperium programming language - IPL
Post by: newbrain on January 25, 2023, 09:31:34 pm
Quote
specification for C
ISO/IEC 9899:2011  5.1.2.3, 5.1.2.4
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 09:38:46 pm
So tell me, what is a "memory model" for, what purpose does it serve? what would happen without one? how many types are there? lots of questions there for you, lets see if you can answer any.

[url=https://en.wikipedia.org/wiki/Memory_model_(programming)]https://en.wikipedia.org/wiki/Memory_model_(programming) (https://en.wikipedia.org/wiki/Memory_model_(programming))[/url]

No surely you mean:
... <waste of his life omitted>
Please show me what the C language standard actually says about a "memory model" after all it now "has" one, it got it "recently".

I think everybody here means the ordering of memory fetch and store operations, as described in the Wikipedia article and in your last link. Yes, your last link is for C++, but for C the semantics are similar. For C, it was introduced in C11. See Wikipedia article.

Quote from: Wikipedia article
The Java Memory Model was the first attempt to provide a comprehensive threading memory model for a popular programming language.[2] After it was established that threads could not be implemented safely as a library without placing certain restrictions on the implementation and, in particular, that the C and C++ standards (C99 and C++03) lacked necessary restrictions,[3][4] the C++ threading subcommittee set to work on suitable memory model; in 2005, they submitted C working document n1131[5] to get the C Committee on board with their efforts. The final revision of the proposed memory model, C++ n2429,[6] was accepted into the C++ draft standard at the October 2007 meeting in Kona.[7] The memory model was then included in the next C++ and C standards, C++11 and C11.[8][9]

Precisely; beat me to it :) It isn't what you can do with it, it is what you can't do without it. Very different. Very important.

The OP has a habit of not reading pointers given to him, apparently preferring to spend his time ranting and casting doubt on the competence of those that nudge him in a useful direction.

He seems to have ignored his stated intention of...
I'm leaving this thread due to excessive trolling and spamming, should anyone want to contribute suggestions or questions about this work please do so using Github issues/discussions (https://github.com/Steadsoft/imperium) which I'll respond to regularly.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 10:05:52 pm
The OP has a habit of not reading pointers given to him, apparently preferring to spend his time ranting and casting doubt on the competence of those that nudge him in a useful direction.

That post itself is a rant and casting doubt on the competence of others. You also never answered any of the questions you were asked, not the first time either, you have habit of not answering questions on topics you don't have a clue about yet pretend to.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 10:10:10 pm
Quote
specification for C
ISO/IEC 9899:2011  5.1.2.3, 5.1.2.4

Right thanks. So if I were to say the design decisions pertaining to this must be deferred until code generation is looked into, would that put a stop to some of incessant prattling some in this thread have resorted to?

Memory consistency models are an integral part of the LLVM suite (https://plv.mpi-sws.org/llvmcs/paper-full.pdf). I fully intend to explore that in due course.

Title: Re: The Imperium programming language - IPL
Post by: DiTBho on January 25, 2023, 10:10:33 pm
The OP has a habit of not reading pointers given to him, apparently preferring to spend his time ranting and casting doubt on the competence of those that nudge him in a useful direction.

Behind write-only mode, well an old saying goes.. after a while arrogance makes the blind  :-//

Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 10:13:47 pm
The OP has a habit of not reading pointers given to him, apparently preferring to spend his time ranting and casting doubt on the competence of those that nudge him in a useful direction.
Behind write-only mode, well an old saying goes.. after a while arrogance makes the blind  :-//

I'll translate that from British English to American English:

Quote
I have nothing constructive to say or offer with respect to designing a new programming language so I'll say something derogatory about the poster because that's all I'm capable of anyway.

Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 10:32:32 pm
Quote
specification for C
ISO/IEC 9899:2011  5.1.2.3, 5.1.2.4

Right thanks. So if I were to say the design decisions pertaining to this must be deferred until code generation is looked into, would that put a stop to some of incessant prattling some in this thread have resorted to?


There's no point in debating syntax and grammar without a shared understanding of what the syntax and grammar constructs mean.

Humpty Dumpty, via Lewis Carroll, had something to say about that back in the mid 19th century.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 10:47:34 pm
Quote
specification for C
ISO/IEC 9899:2011  5.1.2.3, 5.1.2.4

Right thanks. So if I were to say the design decisions pertaining to this must be deferred until code generation is looked into, would that put a stop to some of incessant prattling some in this thread have resorted to?


There's no point in debating syntax and grammar without a shared understanding of what the syntax and grammar constructs mean.

Humpty Dumpty, via Lewis Carroll, had something to say about that back in the mid 19th century.

That's a sweeping statement, rather typical of many posted in this thread. Also note the rather obvious oxymoron too.

How can one discuss anything in that case? we use language to discuss things so how can we discuss meaning of language using language unless we know the meaning of the language? These are old philosophical questions, not new.

Here's one of many open questions I'm dealing with now:

1. Should string literals be enclosed in " or ' characters? (PL/I used ' and C as we know, uses ")
2. Should we support two kinds of string types a basic single byte "text" type and a Unicode capable two-byte "string" type?
3. Should we copy what C# does where ' is used for enclosing a character and " is used for strings?

There are a lot of open questions like this, decisions must be made so that this entire front end phase can begin to wind down.

Are you truly of the opinion that these questions cannot be discussed here and now?

Title: Re: The Imperium programming language - IPL
Post by: ataradov on January 25, 2023, 10:56:15 pm
All those questions are trivial to answer and are not important at all. All of them can be changed once you have anything working at all. This the color of the bike shed discussion all over again.

Also, 2-byte strings for Unicode is a bad idea. Either use plain UTF-8 or UTF-32. There is no point in using 16-bit encodings, you  will instantly be running into the limitation of that when you want to use emojis.
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 25, 2023, 11:05:05 pm
I'm leaving this thread due to excessive trolling and spamming, should anyone want to contribute suggestions or questions about this work please do so using Github issues/discussions (https://github.com/Steadsoft/imperium) which I'll respond to regularly.

Promises.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 11:06:53 pm
Are you truly of the opinion that these questions cannot be discussed here and now?

All those questions can indeed be discussed. The discussions and answers are as interesting and important as which type of car should be in my driveway.

I can use any car, provided I know what the icons on the controls mean. Beyond that the differences are boring and irrelevant. But then my attitude is that there are only a few types of cars: sports, estates, people carriers, vans and lorries. I suppose I ought to update that with ICE, BEV, hybrids. In any case, there is  common factor: too many wheels and engines, too few wings.

Hint: the interesting discussion is about what a language/vehicle is most suited to accomplish, and how.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 11:10:28 pm
I'm leaving this thread due to excessive trolling and spamming, should anyone want to contribute suggestions or questions about this work please do so using Github issues/discussions (https://github.com/Steadsoft/imperium) which I'll respond to regularly.

Promises.

Car crash TV or border police TV programmes are addictive, aren't they :)

Was it you or someone else that was trying to find a way to ignore his thread ;)
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 11:21:57 pm
All those questions are trivial to answer and are not important at all. All of them can be changed once you have anything working at all. This the color of the bike shed discussion all over again.

In which case so is the question of concurrency semantics then yes? trivial! just use the LLVM model which is even better than C11's model, there, done, move on - how's that sound?

Whether a decision be "trivial" or not is a matter of opinion and besides, the matter must be decided along with many others, in order to begin writing test source files.

A seemingly small syntax change can often break a grammar, it can introduce unexpected, undiagnosed inconsistencies, grammars are self referential you know.

Also, 2-byte strings for Unicode is a bad idea. Either use plain UTF-8 or UTF-32. There is no point in using 16-bit encodings, you  will instantly be running into the limitation of that when you want to use emojis.

You might be right about the Unicode. C# used 2 bytes initially and they still do but are now also using a "Rune (https://learn.microsoft.com/en-us/dotnet/api/system.text.rune?view=net-7.0)" data type.

But note that UTF-8 etc are not how strings are stored in memory at runtime. UTF-8 is an encoding, a way of storing a Unicode string in a byte array or file. Some UTF-8 characters are two or three bytes long, but that is not how they are stored in memory in string variables.

So this could mean we want single byte strings (akin to C strings) as well as 2-byte and possibly 4 byte too. So do we have three string type names now "text" string" and "unicode" (say)?

Or might this suggest we consider a different way of describing these types?

Code: [Select]
dcl user_name string (32,1); // 2nd arg optional and must be 1,2 or 4
dcl user_name string (32,2);
dcl user_name string (32,4);

The matter is important because as you well know strings and aggregates of strings can eat memory for breakfast, so having that control is important in a resource constrained setting.

The 1-byte string is absolutely essential I think, many applications will not need anything beyond ASCII so let them keep memory use down. Some might need Unicode but not more than the most common 2-byte support, and then for those that want the whole shebang they can use 4-byte, but also if the language supports all three they can use a mix, a long table of error messages could be in an ascii, 1byte string whereas other stuff could be in Unicode.

These and other questions are things I really want to put to bed, I am keen to get into other phases like analyzing the ASY and so on, but any code doing that might need rework if the grammar is changed so I just want to get as much of it dine and dusted as I reasonably can.

Finally I want to tell you that you often speak of "get it coded" well the grammar IS CODE it is written in a language itself, I am and have been coding for two months!


Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 25, 2023, 11:24:45 pm
I don't think anybody here has ever designed a grammar

Good lord, the arrogance. How the heck would you know?

I've designed everything from new machine code instructions (which have, incidentally, an internal grammar of the construction of their encoding, not to mention their assembly language form), to file and protocol formats, to yes extensions to programming languages. I haven't yet been arrogant enough to think I'm going to design a from-scratch programming language grammar that others will find preferable to something based on something they are already familiar with.

One thing I learned very long ago is that while a machine can readily parse LR(n) grammars it is very unwise to expect a human reader (especially one different to the writer) to do so, and it's best to stick to LL(1) grammars as much as possible.
Title: Re: The Imperium programming language - IPL
Post by: gf on January 25, 2023, 11:26:33 pm
Also, 2-byte strings for Unicode is a bad idea. Either use plain UTF-8 or UTF-32. There is no point in using 16-bit encodings, you  will instantly be running into the limitation of that when you want to use emojis.

Agreed, rather no point for UTF-16, except for interfacing with Windows API functions. Unfortunately, wchar_t is 16-bit on Windows.
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 25, 2023, 11:29:25 pm
I'm leaving this thread due to excessive trolling and spamming, should anyone want to contribute suggestions or questions about this work please do so using Github issues/discussions (https://github.com/Steadsoft/imperium) which I'll respond to regularly.

Promises.

Car crash TV or border police TV programmes are addictive, aren't they :)

Was it you or someone else that was trying to find a way to ignore his thread ;)

It was and I did.

However once the mechanism was discovered I am also using it to ignore uninteresting topics on my "Show unread posts since last visit" and the ignored topics list turns out to be limited to 100 topics. That limit had now been reached, after five days, and this topic rolled off the end of the list.

Most of the new topics naturally go quiescent within five days, and if I have to re-add this topic to the ignore list once every five days that is at least a considerable improvement.
Title: Re: The Imperium programming language - IPL
Post by: gf on January 25, 2023, 11:35:56 pm
UTF-8 is an encoding, a way of storing a Unicode string in a byte array or file. Some UTF-8 characters are two or three bytes long, but that is not how they are stored in memory in string variables.

And the same applies to UTF-16. It still cannot represent every Unicode codepoint with a single 16-bit word, but some codepoints still require a sequence. Only UTF-32 has one 32-bit word per codepoint in the string.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 11:36:48 pm
I don't think anybody here has ever designed a grammar

Good lord, the arrogance. How the heck would you know?

The remarks about grammar being almost superficial to this undertaking is what led me to express that view, sure it sound arrogant but so does this, one of your earliest posts.

Quote
Sadly, it is becoming more and more apparent that you don't know the first thing about either CPU instruction sets or programming languages, and you won't listen to those who do, so I'm out of this conversation.

If only you'd kept that promise.

I've designed everything from new machine code instructions (which have, incidentally, an internal grammar of the construction of their encoding, not to mention their assembly language form), to file and protocol formats, to yes extensions to programming languages. I haven't yet been arrogant enough to think I'm going to design a from-scratch programming language grammar that others will find preferable to something based on something they are already familiar with.

Bruce, I no longer care, you showed your true colors early on, you're not useful in this discussion, please execute your HOULT instruction and give me some peace.

You regard this "to think I'm going to design a from-scratch programming language grammar that others will find preferable to something based on something they are already familiar with." as being "arrogant" so there we have it, you disapprove of the entire idea and did from the beginning. Designing a new programming language is an act of arrogance? wow, I read you like a book didn't I.

One thing I learned very long ago is that while a machine can readily parse LR(n) grammars it is very unwise to expect a human reader (especially one different to the writer) to do so, and it's best to stick to LL(1) grammars as much as possible.

Good lord, the arrogance (do you really think C and C++ are LL(1) the most revered languages in this thread ?? !!)
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 11:44:38 pm
I'm leaving this thread due to excessive trolling and spamming, should anyone want to contribute suggestions or questions about this work please do so using Github issues/discussions (https://github.com/Steadsoft/imperium) which I'll respond to regularly.

Promises.

Car crash TV or border police TV programmes are addictive, aren't they :)

Was it you or someone else that was trying to find a way to ignore his thread ;)

It was and I did.

However once the mechanism was discovered I am also using it to ignore uninteresting topics on my "Show unread posts since last visit" and the ignored topics list turns out to be limited to 100 topics. That limit had now been reached, after five days, and this topic rolled off the end of the list.

Most of the new topics naturally go quiescent within five days, and if I have to re-add this topic to the ignore list once every five days that is at least a considerable improvement.

ISTR the 100 is a magic number required because the ignoring is expensive within the forum's database.

Fortunately I'm a speedreader, so the length of the unread posts list is something I can tolerate without mechanical assistance.

I'm not entirely sure why I'm still on his thread. Maybe I'm channelling St Jude, the patron saint of lost causes.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 25, 2023, 11:48:13 pm
I don't think anybody here has ever designed a grammar

Good lord, the arrogance. How the heck would you know?

I've been the subject of the OP's arrogance, ignorance and willy waving too. I didn't let I go unchallenged and the OP has avoided challenging my response.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 25, 2023, 11:53:35 pm
I don't think anybody here has ever designed a grammar

Good lord, the arrogance. How the heck would you know?

I've been the subject of the OP's arrogance, ignorance and willy waving too. I didn't let I go unchallenged and the OP has avoided challenging my response.

FYI - You've never said anything challenging.
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 25, 2023, 11:57:12 pm
One thing I learned very long ago is that while a machine can readily parse LR(n) grammars it is very unwise to expect a human reader (especially one different to the writer) to do so, and it's best to stick to LL(1) grammars as much as possible.

Good lord, the arrogance (did you really think C and C++ are LL(1) the most revered language in this thread ?? !!)

They are of course not, and to their detriment. Parts of their grammars are simply awful, especially declaration syntax, confusion between declarations and function-style casts, ambiguity of a sequence of +'s, and of course good old dangling else. C and C++ are prime examples of what I am advising against, but which survive and which other languages are based on because of people's preference for the seemingly familiar.

If you must use them (as almost all of us must) then at least stick as much as possible to the LL(1) parts of them by, for example, always using { }, typedefing things to simplify declarations etc.

The Wirthian languages are much better designed, especially after Pascal.
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 26, 2023, 12:11:02 am
ISTR the 100 is a magic number required because the ignoring is expensive within the forum's database.

That's not something I'd ask the database to do. I'd fetch the ignore list and put it in an in-memory hash table, then check each fetched or about-to-be-displayed topic against the hash table.

Quote
Fortunately I'm a speedreader, so the length of the unread posts list is something I can tolerate without mechanical assistance.

No matter how quickly you can read and ignore, it's going to be 10,000 times slower than PHP / Python / Perl / Ruby / Javascript doing the same.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 26, 2023, 12:27:21 am
ISTR the 100 is a magic number required because the ignoring is expensive within the forum's database.

That's not something I'd ask the database to do. I'd fetch the ignore list and put it in an in-memory hash table, then check each fetched or about-to-be-displayed topic against the hash table.

Quote
Fortunately I'm a speedreader, so the length of the unread posts list is something I can tolerate without mechanical assistance.

No matter how quickly you can read and ignore, it's going to be 10,000 times slower than PHP / Python / Perl / Ruby / Javascript doing the same.

All agreed.

One advantage of not pruning the unread posts list is my favourite concept: serendipity. I also like the way the word rolls off the tongue. Maybe I'll get to visit the country too, Serendib.
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 26, 2023, 12:53:03 am
One advantage of not pruning the unread posts list is my favourite concept: serendipity. I also like the way the word rolls off the tongue. Maybe I'll get to visit the country too, Serendib.

I see every topic once when it is created.

I have more serendipity now because as a result of being able to mark topics to ignore I've now been able to stop ignoring entire forum sections (e.g. things such as "Metrology")
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 26, 2023, 01:26:57 am
Designing a new programming language is an act of arrogance? wow

I'm starting to wonder if it's a severe cognitive impairment in understanding what people are actually saying. Seems to read but also seems unable to understand properly  :-//

Some of it could be due to the topics being subtle and involved, thus not suitable for speedreading nor for quick posts on a forum. But other parts don't fit that.

I'm sure psychologists have several names for the observed behaviour; none of it is unique and we've all seen similar things before.
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 26, 2023, 06:30:46 am
Designing a new programming language is an act of arrogance? wow

I'm starting to wonder if it's a severe cognitive impairment in understanding what people are actually saying. Seems to read but also seems unable to understand properly  :-//

Some of it could be due to the topics being subtle and involved, thus not suitable for speedreading nor for quick posts on a forum. But other parts don't fit that.

I'm sure psychologists have several names for the observed behaviour; none of it is unique and we've all seen similar things before.

A short (probably incomplete) list of people who have aroused the ire of OP, leading to them being variously insulted, told they had nothing to contribute and to go away, are being ignored etc:

ataradov, Siwastaja, brucehoult, tggzzz, Nominal Animal, Mechatrommer, MIS42N, DiTBho, DC1MC, MK14

Looks like a roll of honour to me.

The list of contributors to this thread who are not in the above list would I think be shorter. Some of them have flown under the radar a little while "thank"ing posts critical of OP by people in the above list. And one or two seem to have escaped by virtue of, I suspect, comments that went "whoosh!". One comment implicitly comparing posts on this thread to ChatGPT output springs to mind, Mr SW :-)

I'm having similar issues on reddit at the moment with someone who was using a pseudonym (and account created specifically to pick a fight with me, it seems), and who turns out to be the leader of a well known and respected academic programming group. He's around 65 (not much older than me), but based on his apparent comprehension problems and his position, blog posts, and other information I strongly suspect he has done little or no technical work for many many years and is wholly occupied as a manager, fund/grant raiser (he's very proud of this area), and in general navigating public sector bureaucracies (I won't say which ones).
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 26, 2023, 09:21:14 am
...
The list of contributors to this thread who are not in the above list would I think be shorter. Some of them have flown under the radar a little while "thank"ing posts critical of OP by people in the above list. And one or two seem to have escaped by virtue of, I suspect, comments that went "whoosh!". One comment implicitly comparing posts on this thread to ChatGPT output springs to mind, Mr SW :-)

Yes about comments going "whoosh".

The "thanks" mechanism is terminally broken and should be ignored. Just think of how it is (ab)used by treez/ocset/faringdon :)

Quote
I'm having similar issues on reddit at the moment with someone who was using a pseudonym (and account created specifically to pick a fight with me, it seems), and who turns out to be the leader of a well known and respected academic programming group. He's around 65 (not much older than me), but based on his apparent comprehension problems and his position, blog posts, and other information I strongly suspect he has done little or no technical work for many many years and is wholly occupied as a manager, fund/grant raiser (he's very proud of this area), and in general navigating public sector bureaucracies (I won't say which ones).

There's nothing inherently wrong with "going back" to technical subjects after retirement; I'm doing it myself.

Not having humility about the extent of one's own competence is irritating at any stage of life. Even the very competent fall into the trap. There's a classic syndrome where, to employ all the  clichés, a consultant doctor starts flying and then buys a high performance aircraft (especially a glider), and after not too long crashes it. I'm sure there are other equivalent examples in different fields.
Title: Re: The Imperium programming language - IPL
Post by: MIS42N on January 26, 2023, 09:47:29 am
I don't think anybody here has ever designed a grammar yet the prattlers here glibly dismiss its significance.
On a lighter note, I did design a variant of FORTRAN and called it CASHTRAN, a play on my surname. It was quickly pointed out this was an anagram of TRASHCAN. And there it remained.
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 26, 2023, 10:12:33 am
There's a classic syndrome where, to employ all the  clichés, a consultant doctor starts flying and then buys a high performance aircraft (especially a glider), and after not too long crashes it.

A Bonanza is more traditional. Especially the V tailed ones (which were all there were when this cliche started).

If a glider then probably a Stemme, amirite? Or at least an 18m Ventus cM.

Very sad too when active 747 drivers die executing a botched circuit off a winch cable break.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 26, 2023, 11:27:48 am
There's a classic syndrome where, to employ all the  clichés, a consultant doctor starts flying and then buys a high performance aircraft (especially a glider), and after not too long crashes it.

A Bonanza is more traditional. Especially the V tailed ones (which were all there were when this cliche started).

If a glider then probably a Stemme, amirite? Or at least an 18m Ventus cM.

Very sad too when active 747 drivers die executing a botched circuit off a winch cable break.

Yes, indeed. The accident report uses interesting phrases. He had previously splashed down in the sea. His videos will be missed.

Then again, the top glider pilots have a surprisingly high mortality rate, because they habitually fly close to the edge of the envelope.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 26, 2023, 02:56:55 pm
All those questions are trivial to answer and are not important at all. All of them can be changed once you have anything working at all. This the color of the bike shed discussion all over again.

Also, 2-byte strings for Unicode is a bad idea. Either use plain UTF-8 or UTF-32. There is no point in using 16-bit encodings, you  will instantly be running into the limitation of that when you want to use emojis.

I was wrong in my earlier reply, it turns out .Net does use UTF-16 for its strings, this was a compromise the team made twenty years ago in order to avoid having 4-byte chars.

They could have simply restricted their strings to be 16 bit Unicode values only (known as BMP (https://commons.wikimedia.org/wiki/Category:Unicode_BMP)) but they chose instead to adopt UTF-16. This does complicate how strings are worked with, like counting chars in a .Net string entails checking for 1 or 2 Char pairs etc.

Anyway this now suggest that the new language could support all three variants with a simple new attribute.

Code: [Select]
dcl user_name string(32) ansi;  // 1 byte chars, akin to a C char[]
dcl user_name string(32) bmp; // 2 byte chars any Unicode char in the range 0000–FFFF
dcl user_name string(32) ucs;  // 4 byte chars any unicode char whatsoever in the range 000000-10FFFF

These all have fixed length characters of 1, 2 and 4 bytes respectively, easy and fast to index through them to get at character 'I', compute lengths and so on.

But there might be scenarios where one wants to use UTF as the storage method for a string, so this at first sight suggest additional attributes of UTF-8 and UTF-16 in which case space is conserved but some characters might be 2 or 3 or 4 bytes and certain operations cost more as a result.

That would create three additional attributes in a string declaration utf-8, utf-16 and utf-32. But utf-32 is no different to UCS - fixed length chars and the entire range supported, so that makes 5 attributes.

Perhaps the default could then simply be UTF-16 line many languages and the user can specify another if they want that.

This gives a good coverage of string support with speed at one end and compactness at the other, this might even be a better way to express this:

Code: [Select]
dcl user_name string(32) raw(8); // fixed length chars of 1 byte.
dcl user_name string(32) utf(16); // encoded variable length chars of 1 or 2 bytes.

Giving five values:
Code: [Select]
raw(8), utf(8), raw(16), utf(16), utf(32) (being identical also raw(32) but we'd not bother to include that attribute).

So raw strings have every character represented by the same size unit, easy to index, iterate the strings and utf strings are encoded less wasted space but more costly to iterate and search. Simply assigning one to another would implicitly convert the string format too.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 26, 2023, 03:47:15 pm
I don't think anybody here has ever designed a grammar

Good lord, the arrogance. How the heck would you know?

I've been the subject of the OP's arrogance, ignorance and willy waving too. I didn't let I go unchallenged and the OP has avoided challenging my response.

I think some hard facts are in order here.

I started this thread hoping that it would be an informal, friendly sharing of ideas, experienced engineering minded people who had ideas and enthusiasm for the concept of a new programming language that could deliver things that C is poor or weak at, a language that could be designed from the outset with MCU related needs in mind too.

Because a programming language and compiler are not for the beginner, because they are rather large projects, broad in scope and depth, I explained that I was no novice and had in fact designed and built a functioning compiler for a large language and I'd written that compiler in C, I pointed readers at Github where that codebase is available to anyone to look at.

From the outset, from the very first response to my opening post, there was discouragement, disdain and contempt at the mere suggestion of doing this.

After me making just two posts (this  (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536722/#msg4536722)and this (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536830/#msg4536830)) this was what people were posting:

Quote
But in that case don't make polls, start doing it.

and

Quote
Right now this has a vibe of "making your own high performance oscilloscope" threads from 10 years ago. Those went for pages and pages with people daydreaming about it and not actually doing anything.

Note how discussing ideas, soliciting opinions, gathering potential requirements and considering features is regarded by him as "not actually doing anything"! The thread, the attempt to discuss IS DOING SOMETHING.

That frankly, set the tone for the thread, put downs, insults, dismissal and character assassination, I was made to feel unwelcome after making just two posts; go and read those posts and reflect on what I say. Some people here really seem to try and make themselves feel good by attacking others, by attacking someone you remove them and their ideas as a threat, it's called "bullying" (https://en.wikipedia.org/wiki/Bullying) and "mobbing" (https://en.wikipedia.org/wiki/Mobbing) and there are several here, read:

Quote from: Wikipeadia
Mobbing, as a sociological term, means bullying of an individual by a group, in any context, such as a family, peer group, school, workplace, neighborhood, community, or online. When it occurs as physical and emotional abuse in the workplace, such as "ganging up" by co-workers, subordinates or superiors, to force someone out of the workplace through rumor, innuendo, intimidation, humiliation, discrediting, and isolation, it is also referred to as malicious, nonsexual, non-racial/racial, general harassment.

If I've insulted anyone, if I've posted bluntly sometimes then recognize it is a reaction to the rudeness and insults that started less than ONE HOUR after I started the thread.

If you think I'm wasting my time, if you think I'm an idiot, if you think there's no prospect of this ever going anywhere then fine, you have that right, but all I ask is you leave the thread, don't post insults and inflammatory comments and whine and complain, just go away, leave me in peace to discuss a subject I'm passionate about with the few people here who have been polite and considerate and helpful - please.
Title: Re: The Imperium programming language - IPL
Post by: Siwastaja on January 26, 2023, 04:16:57 pm
I started this thread hoping that it would be an informal, friendly sharing of ideas,

 :-DD

Yet already one of your first replies, to the first helpful comments, from ataradov, one of the most helpful forum members who I have never seen fighting with anyone or discouraging any useful idea, was this:
Quote
Anyway, if you have nothing to contribute then of course, you can stop discussing this subject.

When combined with the initial premise:
Quote
I'm a very experienced software developer and very experienced with C

this didn't play well. Rinse and repeat - the rest is history.

The fact you can't deal with anyone here is quite revealing. Maybe take a good look in the mirror?
Title: I don't appreciate being misquoted in a rant
Post by: tggzzz on January 26, 2023, 04:25:22 pm
For the avoidance of doubt, while I did post the first quote (and stand by it), I did not post the other quotes. Hence my editing the thread name.

If anybody is interested in "thread archeology", they might find these posts relevant:
and the observant will note the OP has "repurposed" the thread, not that that matters too much.


I don't think anybody here has ever designed a grammar

Good lord, the arrogance. How the heck would you know?

I've been the subject of the OP's arrogance, ignorance and willy waving too. I didn't let I go unchallenged and the OP has avoided challenging my response.

I think some facts are in order here.

I started this thread hoping that it would be an informal, friendly sharing of ideas, experienced engineering minded people who had ideas and enthusiasm for the concept of a new programming language that could deliver things that C is poor or weak at, a language that could be designed from the outset with MCU related needs in mind too.

Because a programming language and compiler are not for the beginner, because they are rather large projects, broad in scope and depth, I explained that I was no novice and had in fact designed and built a functioning compiler for a large language and I'd written that compiler in C, I pointed readers at Github where that codebase is available to anyone to look at.

From the outset, from the very first response to my opening post, there was discouragement, disdain and contempt at the mere suggestion of doing this.

After me making just two posts (this  (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536722/#msg4536722)and this (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536830/#msg4536830)) this was what people were posting:

Quote
But in that case don't make polls, start doing it.

and

Quote
Right now this has a vibe of "making your own high performance oscilloscope" threads from 10 years ago. Those went for pages and pages with people daydreaming about it and not actually doing anything.

Note how discussing ideas, soliciting opinions, gathering potential requirements and considering features is regarded by him as "not actually doing anything"! The thread, the attempt to discuss IS DOING SOMETHING.

That frankly, set the tone for the thread, put downs, insults, dismissal and character assassination, I was made to feel unwelcome after making just two posts; go and read those posts and reflect on what I say.

If I've insulted anyone, if I've posted bluntly sometimes then recognize it is a reaction to the rudeness and insults that started less than ONE HOUR after I started the thread.

If you think I'm wasting my time, if you think I'm an idiot, if you think there's no prospect of this ever going anywhere then fine, you have that right, but all I ask is you leave the thread, don't post insults and inflammatory comments and whine and complain, just go away, leave me in peace to discuss a subject I'm passionate about with the few people here who have been polite and considerate and helpful - please.

For the avoidance of doubt, while I did post the first quote (and stand by it), I did not post the other quotes. Hence my editing the thread name.

If anybody is interested in "thread archeology", they might find these posts relevant:
and the observant will note the OP has "repurposed" the thread, not that that matters too much.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 26, 2023, 04:27:25 pm
I started this thread hoping that it would be an informal, friendly sharing of ideas,

Yet already one of your first replies, to the first helpful comments, from ataradov, one of the most helpful forum members who I have never seen fighting with anyone or discouraging any useful idea, was this:
Quote
Anyway, if you have nothing to contribute then of course, you can stop discussing this subject.

When combined with the initial premise:
Quote
I'm a very experienced software developer and very experienced with C

this didn't play well. Rinse and repeat - the rest is history.

The fact you can't deal with anyone here is quite revealing. Maybe take a good look in the mirror?

What you say is untrue, the facts prove it, it is another insinuation with slimy words like "one of your first replies, to the first helpful comments, from ataradov" a blatant mischaracterization of the truth.

Yes I did post that, but when? I'll tell you it was in my 14th post AFTER I'd been insulted with this in the 6th post.

Quote
Right now this has a vibe of "making your own high performance oscilloscope" threads from 10 years ago. Those went for pages and pages with people daydreaming about it and not actually doing anything.

That is not a "helpful comment" and you do not speak the truth.


Title: Re: The Imperium programming language - IPL
Post by: Siwastaja on January 26, 2023, 04:32:05 pm
my 14th post

It is interesting that even posts made by others are "your posts".

But I totally get it. Your thread, your rules, other people are yours, too. YOU'RE FIRED IGNORED!!!1 and so on.  :popcorn:

And all ataradov did was to rectify the false ideas about the importance and mechanisms of NOP insertion. Completely helpful.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 26, 2023, 04:39:05 pm
I started this thread hoping that it would be an informal, friendly sharing of ideas,

Yet already one of your first replies, to the first helpful comments, from ataradov, one of the most helpful forum members who I have never seen fighting with anyone or discouraging any useful idea, was this:
Quote
Anyway, if you have nothing to contribute then of course, you can stop discussing this subject.

When combined with the initial premise:
Quote
I'm a very experienced software developer and very experienced with C

this didn't play well. Rinse and repeat - the rest is history.

The fact you can't deal with anyone here is quite revealing. Maybe take a good look in the mirror?

What you say is untrue, the facts prove it, it is another insinuation with slimy words like "one of your first replies, to the first helpful comments, from ataradov" a blatant mischaracterization of the truth.

Yes I did post that, but when? I'll tell you it was in my 14th post AFTER I'd been insulted with this in the 6th post.

Quote
Right now this has a vibe of "making your own high performance oscilloscope" threads from 10 years ago. Those went for pages and pages with people daydreaming about it and not actually doing anything.

That is not a "helpful comment" and you do not speak the truth.

Sijwasta didn't make that comment about scopes; ataradov did. It would be better if you didn't mis-attribute statements.

Even if you dislike that part of the comment, after 1052(!) posts in this thread I don't think it was too far off the mark :)
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 26, 2023, 04:45:03 pm
Sijwasta didn't make that comment about scopes; ataradov did.

I never said otherwise.

It would be better if you didn't mis-attribute statements.

I didn't.


Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 26, 2023, 04:47:45 pm
my 14th post

It is interesting that even posts made by others are "your posts".

But I totally get it. Your thread, your rules, other people are yours, too. YOU'RE FIRED IGNORED!!!1 and so on.  :popcorn:

And all ataradov did was to rectify the false ideas about the importance and mechanisms of NOP insertion. Completely helpful.

Yet despite your false claims I actually did make the 14th post Siwastaja here it is. (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536938/#msg4536938)

The insult that led to me to make the 14th post came before, in the 6th post (less than one hour after me starting the thread) here it is (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536851/#msg4536851).



Title: Re: The Imperium programming language - IPL
Post by: Siwastaja on January 26, 2023, 05:05:27 pm
Yet despite your false claims I actually did make the 14th post

Oh, but your claim was that it was your 14th post, which is only true under the premise that all posts in a thread started by you are "yours".

In actual real world where others do not belong to you, post #14 was your 6th post (including the opening), or your 5th reply.

Or then again, there is the possibility of, to quote Nominal Animal, me failing English.

I can totally feel the importance of this discussion!
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 26, 2023, 05:09:30 pm
my 14th post

It is interesting that even posts made by others are "your posts".

But I totally get it. Your thread, your rules, other people are yours, too. YOU'RE FIRED IGNORED!!!1 and so on.  :popcorn:

And all ataradov did was to rectify the false ideas about the importance and mechanisms of NOP insertion. Completely helpful.

Yet despite your false claims I actually did make the 14th post Siwastaja here it is. (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536938/#msg4536938)

The insult that led to me to make the 14th post came before, in the 6th post (less than one hour after me starting the thread) here it is (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536851/#msg4536851).

Ataradov's post was a calm statement of his tentative opinion. You may not have liked his opinion, but that is a different issue.

You have a tendency, in this thread at least, of being too ready to interpret a statement of a differing opinion as being an insult. (Hint: that's an opinion, not an insult!)
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 26, 2023, 05:15:17 pm
Yet despite your false claims I actually did make the 14th post

Oh, but your claim was that it was your 14th post, which is only true under the premise that all posts in a thread started by you are "yours".

In actual real world where others do not belong to you, post #14 was your 6th post (including the opening), or your 5th reply.

Or then again, there is the possibility of, to quote Nominal Animal, me failing English.

Nah. It is yet another comprehension failure - but not on your part.

IMNSHO the OP would benefit from re-reading a post and his response before posting his response.
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 26, 2023, 05:16:52 pm
Yet despite your false claims I actually did make the 14th post

Oh, but your claim was that it was your 14th post, which is only true under the premise that all posts in a thread started by you are "yours".

In actual real world where others do not belong to you, post #14 was your 6th post (including the opening), or your 5th reply.

Or then again, there is the possibility of, to quote Nominal Animal, me failing English.

I can totally feel the importance of this discussion!

My sincere apologies, I mistakenly wrote "my 14th post" I meant "the 14th post". My 14th post (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4537475/#msg4537475) was absolute post 29.

Just to be very clear then, the 6th post  (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536851/#msg4536851)submitted less than an hour after the thread started was an insult, a disparaging remark directed at me. The 14th post (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536938/#msg4536938) was my response to that insult.

Those insults and disparaging and demeaning comments began at the 6th post and have continued unabated (with a small number of notable exceptions) for two months.





Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 26, 2023, 05:19:15 pm
Nah. It is yet another comprehension failure - but not on your part.

I made a simple mistake and just apologized for it - but as usual, this is seen as an opportunity give me another kick in the face another insult "comprehension failure" and on and on it goes...
Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 26, 2023, 05:30:04 pm
my 14th post

It is interesting that even posts made by others are "your posts".

But I totally get it. Your thread, your rules, other people are yours, too. YOU'RE FIRED IGNORED!!!1 and so on.  :popcorn:

And all ataradov did was to rectify the false ideas about the importance and mechanisms of NOP insertion. Completely helpful.

Yet despite your false claims I actually did make the 14th post Siwastaja here it is. (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536938/#msg4536938)

The insult that led to me to make the 14th post came before, in the 6th post (less than one hour after me starting the thread) here it is (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536851/#msg4536851).

Ataradov's post was a calm statement of his tentative opinion. You may not have liked his opinion, but that is a different issue.

You have a tendency, in this thread at least, of being too ready to interpret a statement of a differing opinion as being an insult. (Hint: that's an opinion, not an insult!)

He said
Quote
This is plain stupid

and

Quote
don't make polls, start doing it and if you have something useful implemented, we can discuss.

and

Quote
Right now this has a vibe of "making your own high performance oscilloscope" threads from 10 years ago. Those went for pages and pages with people daydreaming about it and not actually doing anything.

I have no idea whether he was calm or not, I made no suggestion he was not calm. The post and its tone was an insult, here's the test would a reasonable professional person speak that way to a coworker in a team setting in front of others? in front of say a HR staff member?

I really shouldn't have to even explain this, its just so basic that I'm dumbfounded at your support for it.





Title: Re: The Imperium programming language - IPL
Post by: Sherlock Holmes on January 26, 2023, 05:31:30 pm
Anyway, I'm done with off topic posts and insults, carry on if you must but it's not going to deter me from continuing these discussion with the few individuals that have earned my respect.

Title: Re: The Imperium programming language - IPL
Post by: Siwastaja on January 26, 2023, 07:04:39 pm
Don't you see the difference in saying:

This [doing X] is stupid[, so don't waste your time doing it]
vs.
The person saying this is stupid
?

First is being helpful with no bad intentions, but just without sugar coating. This is discussing matters, not persons. I mean, someone like ataradov writes like that all the freaking time and I'm serious when I'm saying I don't remember it ever causing a misunderstanding before. You're the first I'm seeing having a problem with it, after many years hanging here.

Of course, after enough escalation and actual insults from your part, quite some many of us started really insulting you. You are free to actually ignore everybody as you said you'd do but failed to deliver. You are also free to stop reacting to every little comment you find nasty.
Title: Re: The Imperium programming language - IPL
Post by: magic on January 26, 2023, 07:31:05 pm
I mean, someone like ataradov writes like that all the freaking time and I'm serious when I'm saying I don't remember it ever causing a misunderstanding before. You're the first I'm seeing having a problem with it, after many years hanging here.
Many were triggered but they weren't brave enough to speak up against this horrible abuse.

OP today is starting to sound like Elizabeth Holmes :D
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 26, 2023, 08:02:09 pm
my 14th post

It is interesting that even posts made by others are "your posts".

But I totally get it. Your thread, your rules, other people are yours, too. YOU'RE FIRED IGNORED!!!1 and so on.  :popcorn:

And all ataradov did was to rectify the false ideas about the importance and mechanisms of NOP insertion. Completely helpful.

Yet despite your false claims I actually did make the 14th post Siwastaja here it is. (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536938/#msg4536938)

The insult that led to me to make the 14th post came before, in the 6th post (less than one hour after me starting the thread) here it is (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4536851/#msg4536851).

Ataradov's post was a calm statement of his tentative opinion. You may not have liked his opinion, but that is a different issue.

You have a tendency, in this thread at least, of being too ready to interpret a statement of a differing opinion as being an insult. (Hint: that's an opinion, not an insult!)

He said
Quote
This is plain stupid

and

Quote
don't make polls, start doing it and if you have something useful implemented, we can discuss.

and

Quote
Right now this has a vibe of "making your own high performance oscilloscope" threads from 10 years ago. Those went for pages and pages with people daydreaming about it and not actually doing anything.

I have no idea whether he was calm or not, I made no suggestion he was not calm. The post and its tone was an insult, here's the test would a reasonable professional person speak that way to a coworker in a team setting in front of others? in front of say a HR staff member?

I really shouldn't have to even explain this, its just so basic that I'm dumbfounded at your support for it.

If that's what triggers you, then I am surprised you have survived in a corporate environment!

As someone another person on your ignore list observed...

Don't you see the difference in saying:
This [doing X] is stupid[, so don't waste your time doing it]
vs.
The person saying this is stupid
?

Many many of my statements have been called stupid, and I've called other people's statements stupid. In all cases the comments were right, and absolutely no offence was taken. There may of been to-and-fro establishing why the statements were stupid, but that's all part of coming up with new inventions.

As I taught my daughter (as far as I can see to her benefit), 99% of ideas turn out to be silly, but you need to be prepared to have them in order to discover the 1% that aren't.
Title: Re: The Imperium programming language - IPL
Post by: tggzzz on January 26, 2023, 08:04:51 pm
Anyway, I'm done with off topic posts and insults, carry on if you must but it's not going to deter me from continuing these discussion with the few individuals that have earned my respect.

Are those people that haven't examined, questioned, and challenged your statements? If so, then that's merely an echo chamber or mutual-support group.
Title: Re: The Imperium programming language - IPL
Post by: brucehoult on January 26, 2023, 08:20:32 pm
Giving five values:
Code: [Select]
raw(8), utf(8), raw(16), utf(16), utf(32) (being identical also raw(32) but we'd not bother to include that attribute).

So raw strings have every character represented by the same size unit, easy to index, iterate the strings and utf strings are encoded less wasted space but more costly to iterate and search. Simply assigning one to another would implicitly convert the string format too.

Lovely!

According to nextpcb.com, one of the most used microcontrollers in 2022 was the PIC 16F84, with 1024 instructions of ROM and 68 bytes of RAM.

It's going to love 5x duplicated string libraries.

Don't make any decisions ... just include every possible option! This is indeed the true spirit of PL/I and Algol68, not to mention the CPL.

Thank God for Martin Richards, Niklaus Wirth, and Dennis Ritchie is all I can say. Half a century on they still look more practical than their (would be) successors.