Making A Programming Language


mickb22

Still Fresh
Joined
Sep 10, 2008
Messages
57
Age
42
Website
Visit site
I'm just curious, but what does a programming language run off of, or like how does everything you type in do the task?
does someone program a programming language with another one?
how is that one programmed?

i'm thinking and i think it might be binary or, ASCII, or that one lanuage, umm , hex- something.
just curious do any of you know?
 
The first programming languages were written in raw binary and later Assembly. (Assembly just has a short command assigned to the bit pattern of each instruction which is called a mnemonic.)

If I were going to write a new language now, (and I currently am) I'd use LLVM as my Assembly language since it will translate efficiently to other binary formats. This means you can write a program in LLVM Assembly and compile it to PowerPC, Intel x86, or ARM or others without having to rewrite it. It's conceptually similar to Java bytecode or the MSIL that .NET is written in, but easier to work with.
 
Hexadecimal is just another way of showing the binary. It's easier to read it in hex than to stare down a bunch of ones and zeroes.


edit: I just realized that the question was more about designing languages than building a compiler. Read on if you want to be misinformed :(
edit again: it was samurai crow who mentioned llvm, typo on my part.

Samurai_Crow mentioned that some are written in assembly. There are other ways, too. I believe that the Python interpreter was written in C. But there are other implementations of Python that exist, like PyPy. Java libraries are written in Java or C/C++, and there is at least one Java VM that was written in Java. I think most C compilers end up being written in C as well.

If you're writing a compiler or interpreter, it doesn't matter what you write it in, as long as it does what you want. Obviously writing a C compiler in C leaves you with a chicken-and-egg problem, namely, where did the first one come from? Llike crow said, it goes back to assembly. In the case of C it wasn't x86. It was originally on a PDP machine, I don't know which exact one. But that doesn't mean that all compilers, etc end up being done in assembly these days. From what I understand, most of them are written in the language that they are supposed to be compiling.

note: assembly and machine code are very close. on the oldest machines, assemblers were not always available. these days though, i don't think there are many platforms for which someone wouldn't use an assembler if they wanted to write at that level.
 
the end result of a program is really know different from each other. programming languages don't matter except in the development process- the end result is just binary. programming languages are just made from assembly, usually, or low level programming languages and convert their syntax to binary.
 
PoisonedV said:
the end result of a program is really know different from each other. programming languages don't matter except in the development process- the end result is just binary. programming languages are just made from assembly, usually, or low level programming languages and convert their syntax to binary.
Python was written in C and I believe Ruby was too. C is really low or high level depending on the conversation, but I'd still call it high-level.

I think the most important thing is the conception of the language (what features will it have, what syntax, etc). I say this because, for an given language, there is probably an implementation in a couple more, including the original language. Like Jython, PyPy, JRuby, MetaRuby, IronPython, etc.
 
Last edited by a moderator:
Samurai_Crow said:
The first programming languages were written in raw binary
You know, I'd HATE to have that job.
 
Last edited by a moderator:
I'm currently writing a sort of scripting language called O-Script for use with a MUDD I'm developing. The language is written in PHP/MySQL and will be used to control the whole environment.

Some languages are written in an editor and uses a separate resource to compile them in to executables. The downside of this is the program will only run on what it was compiled for, such as something compiled for windows will not run on Linux.

Some languages use a runtime that interoperates the program as it is running, the practical upshot of this is that each platform have their own runtime but uses the same source program, but because it is being compiled live (So to speak), execution is slower than a pre-compiled program. An example of this is games written in Fenix, the same program will run on multiple platforms such as Windows, Linux, GP2X etc as each have their own version of the runtime.
 
rokdcasbah said:
edit: I just realized that the question was more about designing languages than building a compiler. Read on if you want to be misinformed :(
Nah, I think your answer about a compiler is relevant

"like how does everything you type in do the task?" - by compiling it into machine code :)

besides, you don't really 'program' a language, you program a compiler/interpreter to convert/run that language (in the broadest sense..no picky details please). The language can be designed in your head, on a bit of paper, some sort of flow chart...

see these from wikipediea (a bit long winded)

http://en.wikipedia.org/wiki/Programming_language
http://en.wikipedia.org/wiki/Compiler
 
Last edited by a moderator:
Two ways.

1. Write a program that takes code written in the programming language you invent, and executes its instructions.

2. Compile (convert) the programming language code into native machine code, which the computer can run directly.

3. You can convert the language code into some other language (or bytecode) that a different program can execute. Shit, that's three ways!

As for what all these extra programs are written in? Your choice. Typically C is used.

Problem is, writing a compiler to native machine code is incredibly difficult to get right. There are lots of annoying things to consider. That's why modern programming languages tend to go with option 1 (which is easy; Ruby 1.8 does this, Perl does this I think) or option 3 (e.g. all the .NET languages convert down to the same bytecode, which the .NET CLR runs).

If you're interested in writing an interpreter or compiler, I recommend "Modern compiler implementation in C" by Appel.
 
QUOTE
I'm just curious, but what does a programming language run off of, or like how does everything you type in do the task?
does someone program a programming language with another one?
how is that one programmed?

i'm thinking and i think it might be binary or, ASCII, or that one lanuage, umm , hex- something.
just curious do any of you know?


Well, I deliberately skipped my university's "Compilers & Interpreters" course in the third-year after a single lecture - it was an incredibly dull and complicated subject. This is an extremely simplified explanation.

Basically, a compiler is a program like any other that just takes some text and converts it to machine code (actual 1's and 0's that the CPU runs directly - you might know it better as an "executable" or "program") and then saves it somewhere.

When you want to write, say, C code and put it on a computer, you use a C compiler. It "converts" the C to machine code that the CPU just runs. Note that usually this compiler DOES NOT run on the CPU that you are aiming at - most GP2X software is *COMPILED* on a normal PC with a compiler that outputs "ARM" code (the type of chip in the GP2X) - but it *OUTPUTS* machine code for the chip that you want (called cross-compiling). So a PC compiler will output "ARM" code that you can transfer to an ARM device somehow (usually by putting it into a memory chip) and make it run.

But who writes the compiler, I hear you ask? That's an incredibly complicated program that is usually only written once for each new type of "chip" - GCC (the most-common collection of various compilers used in open-source software) has a compiler for ARM, one for x86 ("normal" PC chips), one for 68000, and so on. Writing a compiler is a notoriously difficult task that requires a complete knowledge not only of every quirk of the programming language itself but of the target chip.

The programming language C exists because in the past "porting" a compiler to another type of machine was a time-consuming task that was usually done in something very, very close to machine code (literally, you are giving the computer the exact bytes that it should put into its CPU for every cycle). When early-UNIX was being ported to another chip, the language C was invented - it's a real programming language but was written with this exact purpose in mind. C is a very low-level language (pedants: I'm simplifying, not using the real definition of high/low level) - converting from C to actual machine code is a quite trivial process (relatively speaking). So *writing* a program that can take in C "code" and produce machine code for a new chip tends to be quite simple.

This allows people to do nothing more than create a C compiler for a new "chip" and they can then compile the C code for all the important parts (e.g. Linux is written in C, compilers for other languages are usually written in C, device drivers are usually written in C, etc.) and then it automatically "works" on the new chip (It's not quite as easy as it sounds, but that's the gist of it).

Modern compilers, even C compilers like GCC, tend to be written in C itself (yes, I know, it sounds absolutely crazy), so when a new type of "chip" is made, the only thing that *needs* to be written for it is a very, very primitive C compiler with the most basic of features. You can use that to compile a "real" C compiler for the new chip, and that compiler has EVERYTHING that's in C. Then you can use that "full" C compiler to compile one of the myriad other programming languages and use *those* to actually write software for the machine.

So basically:

- New Chip
- Write a "Simple" C Compiler to turn "simple" C into "New Chip" Machine Code (this is usually written in Machine Code itself)
- Convert a "Full" C Compiler to Machine Code using this simple compiler.
- Use the "Full" C Compiler to turn "Full" C in Machine Code (Other Compilers, Linux, Windows, Device Drivers, the rest of the software, etc.)

Steps are sometimes missed out (e.g. sometimes a "Full C" compiler is written straight away, sometimes the new chips are similar to the old ones or have compatibility with them, so you don't need to write new compilers each time, etc.).

To complicate matters, most compilers now are written in C themselves or don't convert directly to machine code (they are technically "interpreters", and need a special program which IS written in machine code to "run" them). So there's a massive chain of software which depends on a massive chain of various compilers but which ultimately culminates in some poor sod somewhere hand-writing a small compiler in native machine code (literally 0's and 1's but we tend to use "assembly" instead, which is just a human-readable symbolic representation of those 0's and 1's). This is typically done exactly once for each type of chip and typically the person to do this would be, for example, someone who works for the chip manufacturer (e.g. ARM, Intel etc.) or someone who works for a compiler-supplier (e.g. Microsoft, Borland, GCC, etc.). This is why Intel have a (very expensive) compiler. Some "new" chips come with their own C compiler that's been written by the chip manufacturer, etc.

Writing a compiler is quite possibly the most horrible job in the IT world. Writing one in machine code for a chip that you don't know is worse. But somewhere, every time a new type of chip is released, some poor sod ends up doing it. Nowadays, new *types* of chips are rare but they are still produced. Most new chips are based on similar older chips so getting that all important first-compiler is just a matter of using one that already exists for the old chips.

Eventually, though, you will trace everything back to some poor sod coding directly a compiler in "machine code".
 
As someone who has written a compiler, I would argue that it's not quite as hard as say, multi-threaded programming. A couple notes:
  • A compiler is merely a program that converts one language to another. It doesn't have to be machine code.
  • Modern compilers such as GCC have a 'frontend' that converts the source language to an intermediate representation that is language-agnostic, and a 'backend' that converts that representation to the target language.
Consider we have four languages - Java, C, Pascal, Fortran - and four targets - x86, x86-64, mips, arm. If we wrote compilers for every combination of language and target we'd need 16 compilers!
Instead, we write four frontends and four backends, and design one intermediate representation that can handle everything that any of those languages would want to do (in a simplified way). This saves us a LOT of time.

edit: So due to what I said above, the 'poor sod' doesn't need to write a compiler in native machine code for each new chip type. He just writes a backend to GCC, typically in C. Usually this backend is a subtle copy of an existing one.
 
Samurai_Crow said:
If I were going to write a new language now, (and I currently am) I'd use LLVM as my Assembly language since it will translate efficiently to other binary formats. This means you can write a program in LLVM Assembly and compile it to PowerPC, Intel x86, or ARM or others without having to rewrite it. It's conceptually similar to Java bytecode or the MSIL that .NET is written in, but easier to work with.



Further to this, the LLVM docs have a nice tutorial that details (surprise!) how to implement a new language. They invent a toy one called "Kaleidoscope" that is still flexible enough to compute ASCII representations of the Mandelbrot set :)

http://llvm.org/docs/tutorial/LangImpl1.html

Very interesting technology, LLVM - I can easily see it surpassing GCC in terms of capability once Clang is more advanced, probably in 3 years or so.
 
Last edited by a moderator:
html is a good progam to start with(some would argue that html is not a real progammable langaege) you use hex and can add javascript to it to spice things up. )
anyway, What about fenix languge (sorry for the mispellings) I would love to learn fenix at a later time. What excactly does fenix do? Does anybody know where I could get a fenix book?
 
IIRC, Fenix is a scripting language. There is no book for it. Personally, I would skip Fenix and learn another, more wide spread scripting language such as Python. It is a good language to start learning from and is widely used in both games and the software industry.

There are plenty of libraries and bindings for it such as Pygame and Pyglet, hopefully some of which will later be ported to the Pandora.
 
alterdimension said:
html is a good progam to start with(some would argue that html is not a real progammable langaege) you use hex and can add javascript to it to spice things up. )
anyway, What about fenix languge (sorry for the mispellings) I would love to learn fenix at a later time. What excactly does fenix do? Does anybody know where I could get a fenix book?
HTML is, as its' name suggests, a mark up language.

Hexadecimal is just a nice way to represent big binary numbers and is useful as one hexadecimal digit corresponds to 4 binary bits.

Javascript is indeed an interpreted programming language.

Fenix is an interpreted language designed to make it easy to create games. It is based on the now defunct DIV Games Studio.

alterdimension said:
can you buy a book on python in a local bookstore?
Yes.
 
Last edited by a moderator:
I had a go at writing a basic compiler in ASM in about 1996/7 when i was a crazzzyy young teen.

I had a nightmare of a time working out how to do the maths functions sin, cos, tan etc. in a day before you could look these things up on the net. I never found out how to do one of them, I can't remember which and I do remember the answer was a lot of ^.

I used to have fun trying to match other compilers for speed etc. but I never finished it after about 6 months of 'for fun' work. Christ. I can't believe I did all that back then. Why didn't I just turn off the TV and go outside and do something better instead?

Someone else has always done it already, and thanks to the net these days you can find it all! The future is here!
 
craigix said:
I had a nightmare of a time working out how to do the maths functions sin, cos, tan etc. in a day before you could look these things up on the net. I never found out how to do one of them, I can't remember which and I do remember the answer was a lot of ^.
It's simple really. You don't put them in and leave it to someone else to make a libary :rolleyes: :lol: .

More seriously, I guess you approximate them with polynomials or something? Or is it binomials.... I think there is an omial in there somewhere :S.
 
Last edited by a moderator:
Looking back it was very clever that computers like the Spectrum etc. could do cos, sin, tan etc. and make it look as if the computer could handle it with ease, instead they used clever calculations on their 8bit cpus which gave amazingly accurate answers fast.

I'm in a mood to look it all up again now.
 
Back
Top