MIME types

[dflemstr]

I actually don't think that this discussion is about executables any longer. It seems to have glided to a generic discussion about the MIME vs extension system. Is this really relevant any longer? Since the Pandora uses Ångström, it WILL use MIME-types (or libmagic) wether we like it or not; if someone doesn't make a new file manager just to support file extensions, of course.

If this discussion goes on, it will only be idle talk, since it won't have a 'goal' of sorts.
But sure, this might be fun to discuss.


And to "reply" to some of the things that have been said so far:

Linux is generally seen as substantially more secure than the Redmond OS (No flamewars, please. Just stating what the general opinion seems to be.)
Some ways this shows is through the file handling security features that exist.

For example: An executable in Linux CAN NOT HAVE an icon (that is custom, I mean; of course executables in general have icons), so "veryfunstory.txt" won't work as a trojan. Since everything is recognized by using libmagic, it is almost guaranteed that the file contains what the icon says it does. As opposed to the extension system.
Also, an executable cannot be executed if it doesn't have the executable bit set, so viruses (viri?) in disguise as something else can't be executed by accident. No worries

And yes, the PackageKit thing that I mentioned will of course ask before installing software.


EDIT:
"I know Linux will run an executable regardless of file extension, but thats more of a casual check of the file header ("does first line contain '#!'" and "are the first four bytes 7F 45 4C 46)"

That's libmagic at work, right there.

 

[urjaman]

"I know Linux will run an executable regardless of file extension, but thats more of a casual check of the file header ("does first line contain '#!'" and "are the first four bytes 7F 45 4C 46)"

That's libmagic at work, right there.

NO, That's the kernels executable loader at work. First it checks if its a script - if it starts "#!" then it's a script and the interpreter to execute the script with follows. If it isn't a script then it checks for the other executable loaders - usually ELF (it could be (very rarely now) a.out too)

Also btw, posted from a virtual ARM machine (qemu)

EDIT: fix typos

 

[dflemstr]

NO, That's the kernels executable loader at work. First it checks if its a script - if it starts "#!" then it's a script and the interpreter to execute the script with follows. If it isn't a script then it checks for the other executable loaders - usually ELF (it could be (very rarely now) a.out too)

I actually did not know that! Well, you learn new things every day, and so far I've stayed away from the kernel.
Thought it was shell dependant or something.
That would explain the Wine magic that I've seen recently.

 

[Kramy]

Ubuntu has a special icon for executables, and when you double-click an executable text file like a script, it will open a dialog box that says, "This is an executable text file. Do you want to run it or open it?"

You mean that diamond icon thing?

I see that all over the place when browsing NTFS volumes, and I assure you none are linux executables. :lol:

Well, I for one will not care too much about what you do with extensions vs. mime/libmagic, except this: Whatever you choose to do, please leave out extensions for executables!

I don't agree with your argument. You have one factor: In Linux it isn't done that way. I could argue against you just as easily. In Windows, it isn't done the linux way.

Take this Scenario:

You have Windows. You create games on Windows, for your GP2X/Pandora. After compiling, you wipe out the extensions, because some annoying linux geeks decided the linux-y way of doing it was "better", rather than compromising. Isn't that annoying? Can't even search the damn executables out, because there's no extension! What idiots! :rolleyes:

(I've always been a fan of making stuff work between operating systems. When I wrote my fancy xml parser years ago, I made sure it worked fine with XML files written in text editors from Windows, Linux, and OSX. I didn't put limitations in it like "CR+LF" just because it seemed right. I went for compatibility. It even went so far as to write XML files compatible with your current ones. (technically I went beyond just newlines, and included non-escaped tags, but that's irrelevent) An extension for executables on the SD card is such a small thing, that it leaves me dumbfounded people are arguing against it.)

For example: An executable in Linux CAN NOT HAVE an icon (that is custom, I mean; of course executables in general have icons), so "veryfunstory.txt" won't work as a trojan. Since everything is recognized by using libmagic, it is almost guaranteed that the file contains what the icon says it does. As opposed to the extension system.
Also, an executable cannot be executed if it doesn't have the executable bit set, so viruses (viri?) in disguise as something else can't be executed by accident. No worries

When I copy an executable over from my Windows box to my SD card(via card reader) while debugging, this bit gets automatically set? Seems dangerous.

And yes, the PackageKit thing that I mentioned will of course ask before installing software.

Good.

 

[lulzfish]

Take this Scenario:

You have Windows. You create games on Windows, for your GP2X/Pandora. After compiling, you wipe out the extensions, because some annoying linux geeks decided the linux-y way of doing it was "better", rather than compromising. Isn't that annoying? Can't even search the damn executables out, because there's no extension! What idiots! :rolleyes:

You can tell it to run a .exe, too, you know.
It's not a big freaking deal, we're just saying "don't require any particular extension, just go by the x bit"

And why would you use a different extension when compiling anyway?
Just compile to "main" and then look in Windows Explorer for "main" and there it is, copy it to your Pandora and run.

It's really not a big deal either way.

edit: This really is Windows Explorer's fault for using extensions instead of MIME types, but since libmagic is used in basically all Linux file managers, you don't need to worry about this with the Pandora.

So, again, compile to whatever extension you like. It's not like it's terribly hard to remember, "Oh, did I put that as .exe or .nothing??"

edit again: Sorry if I sound pissed, but you seemed to think extensions were a BFDeal on either platform. On Windows they're just default icons and applications, on Linux they're even less, and the Pandora will run whatever its PXML asks it to.

 

[Kramy]

Sorry if I sound pissed, too.

I just hate seeing something done one way, which complicates another, when it's completely trivial to the first way to compromise.

One of the programs I frequently use on Windows is Locate32.
CODE
http://www.locate32.net/


Using that, with some strict folder and file naming, I can find anything I need on my computer in about 5 seconds. Maybe less. Sometimes it only takes 1-2 seconds. This is with millions of files on my system. Thousands of projects, amassed over the past decade.

Can you in total honesty say that you often navigate around your computer to exactly where you want to go, in about a second? This is a huge time saver, but not having an extension means I'll have to jump through shortcuts or *gasp* navigate through explorer to find a Pandora executable and copy to an SD card.

As an example, if I type in "sokoban", it gives me this: CODE
F:\Programming Software\Sun Java\Apache Tomcat 6.0\webapps\sokoban2k


Most of my folder hierarchies are at least that deep. Without an extension, if I were searching for a Pandora app, I might lose a whole minute getting to it! That's a 30x loss in productivity.

I'm envisioning making small alterations to a game - altering a few lines, compiling the changes, then testing to see how it affects the gameplay - if I can't get it to my Pandora in 15 seconds flat, development time on projects might jump from days to weeks or months. Or I'll have to develop Win32 and Pandora versions in parallel, and do all the important testing on Windows. (Looks like other people already hit this snag, and went that route. )

But to re-iterate. An extension is useful on non-linux.

Oh, and could someone answer the questions I posed that haven't been answered?
-X bit get automatically set on Windows?
-Why is Ubuntu showing random files on my NTFS volumes as executables?

 

[lulzfish]

I can't tell what you're saying, but I usually keep all my stuff in one folder, and I park a terminal or two in that folder.

So compiling [Either on Linux or MSYS] would be something like:
make

or possibly, in its most complex form:

qmake -project
qmake main.pro
make

And then transferring to the Pandora for testing...

Well, my Pandora will have an SSHFS link to my computer, so testing will be:

./main

from the Pandora's terminal.

Um, Windows doesn't really have an X bit, so I think Ubuntu automatically marks everything on an NTFS partition as either non-executable or executable.

So, anything Ubuntu can't recognize on your Windows partition will be "executable", just in case.

I think this either Windows' fault for not supporting POSIX standards or Linux's fault for using non-standard executable determining methods. Who knows... Not me.

 

[Kramy]

Um, Windows doesn't really have an X bit, so I think Ubuntu automatically marks everything on an NTFS partition as either non-executable or executable.

Well if it doesn't support an X bit, what stops someone from renaming a pandora executable to preview.jpg, or lastrun.log?

I unzip the game, toss it on the SD card. In Windows, there's a jpeg and a log file. The Pandora OS decides one of the files is an executable, and displays the game's icon, which matches the icon for that filetype(jpeg, .log, etc.). I click on it to open the jpeg or log on the Pandora, and... I get screwed? :lol: If it was really devious, it could even open the file handler in question. (GIMP, text editor, etc.) with fake data, and I'd never know it just stole my Firefox passwords, or something like that.

I'm honestly asking - what sort of protections are in place to prevent this? If you know, tell me, because I'm not pleased by BS answers like "well linux does it this way so it must be better"

There will be cross-OS situations with the Pandora, and they will be very common. So why should this not worry me?

 

[lulzfish]

I don't think you're really understanding how files are opened in file managers.

*cut this part out, I was just rambling*

To answer your questions more directly:

"I'm honestly asking - what sort of protections are in place to prevent this?"

1. The file manager labels files by what it believes them to be. If you see a "preview.jpg" that doesn't show up as an image file, you should suspect something is wrong.

2. The file manager's default actions always match what it believes a file to be. If you see a "preview.jpg" that's labelled as an image file, the file manager will send it to an image viewer.

3. Basically, the only way to accidentally execute malware disguised as an image file is to ignore the icon label, tell your file manager to execute the file, and then get what you deserve for executing an image.

4. The most important defense is the user's judgement. Don't download stuff from people you can't trust, and don't click on an 'image' if your file manager says it wants to run it as an executable.

"So why should this not worry me?"

Because you shouldn't be downloading malicious software anyway, and because image viewers don't execute any code from the images they read in, regardless of its x bit. The execute bit is just a permission for users, it doesn't magically make data executable by image viewing programs and the like. That would be stupid.

 

[Kramy]

1) .jpg was just an example. Unless you're saying every kind of file can be previewed in Linux? There's lots of other extensions/formats that (I believe) have no preview, which renders this moot? The fallback would be icons.

2) Right.

Valid jpeg image -> GIMP
Fake jpeg image -> Executable -> Open valid jpeg image -> GIMP -> Steal Passwords in background.

Both open GIMP and let you view/edit the image.

3) What prevents executables from having matching icons? In this thread it has been said that there's a unique executable icon to denote executables, but also that executables can have their own icons. (as has been proven with games?)

So which one is it? Are they locked to an ".exe icon", or can they have their own?

4) That would be any upload to the GP32x file archive? Anything posted on these boards? Because that's where malware would show up if someone created it.

QUOTE
and don't click on an 'image' if your file manager says it wants to run it as an executable.

What's this mean? There'll be a dialog warning us it's an executable?

 

[lulzfish]

File managers don't usually have special icons in my experience, the icons are usually restricted to desktop environments and in this case, the Pandora menus.

So, a fake JPEG will show up as an executable, in which case you should either ignore it, point it out to someone, or tell the manager specifically to open it as an image.

No, there's not necessarily a dialog box. But it has an executable icon, so you're sort of expected to assume it will be executed.

In the case of the Nautilus file manager I'm using, images show up as a thumbnail, videos show up as a screenshot, music shows up as a musical note [and you can hover over it to preview], and executables show up with a special "executable" logo.

More basic file managers will probably forego the thumbnails and just have icons based on file type.

Like I said, I don't think application icons are actually supported in native Linux programs. It seems to be a Windows Explorer thing. And this makes sense, because you could disguise a program as an image.

"Fake jpeg image -> Executable -> Open valid jpeg image -> GIMP -> Steal Passwords in background."

The developers have said that there will be a sandbox for most applications to run in. You can keep anything suspicious on an SD card sandbox, where it won't be able to read your passwords or anything.

I'm getting a little lost here. Maybe you could state your point clearly so I can keep us on track?
If you're just asking, "what will protect me?", then:
1. sudo permissions will protect your OS
2. sandbox will protect your personal data
3. MIME types and file manager icons will protect ... the application's folder, I guess.

If you really want to download something and run it, you're going to be running foreign code at some level or another. It's all about trust.

 

[Kramy]

I think you answered my questions in your last post.

I said earlier I don't have much experience with Linux. Ubuntu is pretty much it, and on my computers it often gets icons wrong. Naturally I'm hesitant to rely on them.

On windows I have all exe icons stamped with "exe". It's a visual marker.

I also display all file extensions, but for extremely long filenames, the extension sometimes gets cut off.

If previewing can be totally disabled, and nothing can spoof the executable icon, then all should be fine.

Never heard of this sandbox thing. I'll have to search it out.

 

[lulzfish]

Yeah, Windows Explorer likes to hide default extensions, it's kind of an annoying 'feature'.

 

[Squidge]

Yeah, Windows Explorer likes to hide default extensions, it's kind of an annoying 'feature'.

It is kinda daft. You have a folder with 50 files in it, and the executable icon can be anything. So you click randomly until one of them does something useful. The only alternative being to select "Detail" mode that it'll actually tell you whether or not it's an application, or to enable file extensions so you can tell yourself.

 

[urjaman]

Yeah, Windows Explorer likes to hide default extensions, it's kind of an annoying 'feature'.

Yeah, the (now very rare) times when i use windows i always disable that 'feature'...

 

[dflemstr]

Anyways, to summarize answers to questions that have been asked in the last couple of posts and where the answers weren't neccessarily straightforward (for any lurkers or others that take part in this discussion):
(I actually had to make a NTFS partition to get some of this info )

1. If you mount Windows partitions in Linux, they do by default have +x set on ALL files, including executables. This is to mimic the Windows behavior, where you don't have to set +x before executing a file. This "feature" can be turned off with "noexec", so that all files get permission 6 (read/write), which is a LOT more secure, but you can't execute programs on a windows partition then, but instead have to copy them over.

1.1. Many files that look like scripts or applications on a Windows partition that has been mounted will "look" like they are executables, because the system believes that they are. The system will, however, ask every time when you try to open a file that it thinks is an executable, wether it is or not. Therefore, it is impossible to execute an executable with the .jpg extension (or anything else) by accident, since a dialog will pop up, asking first.

2. Icons are always determined by file types in Linux. If a file is a JPEG file, it will have the JPEG icon, or an image preview as the case may be, even if it has the .exe extension or whatever.

2.1. Executables CANNOT have icons of their own like in windows. A file will always have the icon of it's type, or a preview. Previews almost exclusively exist for images and documents. The sole exception to this rule are .desktop files, that CAN have icons of their own in most file managers. These are almost always links to executables, however, so it's not like you would execute them by thinking they were other files.

3. The Linux system does never trust any user (except root). It is almost impossible for a program, even if it's startedand has +x, to do any harm to the system, because everything not user-controlled is owned by root with very strict permissions.

4. MIME types cannot actually determine that an executable is executable and then execute it as default.
File managers just recognize the file as executable through MIME types, then check the +x-bit, THEN sends the file to the kernel for execution, and the kernel can still deny a file because it's not executable. It is almost impossible to execute something by accident in Linux.

5. The "sandboxing" mentioned is called chrooting, or "changing the root (/) directory". This makes it possible to temporarily limit a program or an user to only be able to access a single directory and it's subfolders, 'locking' away the rest of the system. The program or user can then only do any harm or any good in that folder, and the rest of the system cannot be harmed in any way.

6. Explorer does indeed hide extensions by default. Not the brightest of ideas the Windows dev team has had.

 

[Kramy]

Good summary.

I was aware of #3/4

So #5 means an app can't access any data from other apps? I believe on Windows, even for limited users, programs running off a USB stick would have access to the whole USB stick.


Oh, if you don't like exe files having their own icons in Windows, you can disable them really easily. Open RegEdit and find:
CODE
HKEY_CLASSES_ROOT\exefile\DefaultIcon

It should be:
CODE
%1

Replace it with this:
CODE
%SystemRoot%\System32\shell32.dll,2

Then rebuild icons with TweakUI, or reboot.

 

[dflemstr]

So #5 means an app can't access any data from other apps? I believe on Windows, even for limited users, programs running off a USB stick would have access to the whole USB stick.

You have read access to most things on the system, write access only to your own files most of the time, and no access at all to password files, encryption files, logs etc.
USB sticks are only writable by the one who inserted/mounted it. So, if you inserted the stick and started the program, the program would indeed be able to write to the stick (if you don't have SELinux or somesuch)

And, of course, all of these settings can be changed etc, this is just the default.
All in all, no real impact on the system files if some virus gets in; only your personal files might be in danger. This requres the virus writer to know what kind of files you have, which makes it very unlikely, though.