Blue Net – Support

First some basic cheats, so you don’t have to search through the whole post:

ctrl+a+c                                               make a new screen
ctrl+a+n                                               next screen down the line
ctrl+a+p                                               previous screen down the line
ctrl+a+k                                               kills current screen until last one then you’re out
ctrl+a+d                                               leave, but leave them running and re-attachable

screen -ls                                            31619.something.else (Detached)
screen -r 31619.something.else       re-attaches to that session

screen -DDR                                     to re-attach any (the first) detached session.
If there is none, it creates a new session.

If you’ve ever used SSH to connect to a server, you ‘ll know its limitations: if you want to open a new window, you’ll need to create a second SSH connection to the server. And if the connection breaks during the SSH tunnel, you’ve lost your progress. This is where Screen comes in.

Screen, which calls itself a “full-screen window manager that multiplexes a physical terminal between several processes (typically interactive shells).” is a usefull tool to use 1 SSH connection, but use multiple screens to work in. So you can have 1 screen to write some scripts in, another to tail a logfile and a third to check your IRC messages

Screen is installed by default on most installations, you can verify this by running the “which screen” command. If it’s not installed, try to apt-get or yum it – it’s in most repositories.

You can start screen by typing:

[root@vps ~]# screen

This makes sense, right?
You’ll probably notice that not much happens if you type that. At least, it seems like not much as happened. In fact, you’ve just opened a new “screen” to type your commands. The program “screen” has a few commands of its own, in order to create a new window, and navigate through the open ones.

Once you’ve opened “screen“, you can see a command list by typing “CTRL + A”, followed by “?” (the question mark). By typing “CTRL + A” you state that the next signal is to be sent to the program “screen“, in stead of to the Shell (like you would in a normal shell). You’ll see a list of all bindings known to “screen“.

Start a new window by typing “CTRL + A” + “c”. The C stands for Create – I know, too obvious. A new window will be created. In order to test this, type the command “top”. Then create a new window, by using “CTRL + A” + “c”. You’ll see top disappear, and a new window will open. Type in some commands of your choice, and return to the previous window, by doing “CTRL + A” + “n”. The “n” stands for “Next”, and will open the next screen. “CTRL + A” + “p” would’ve opened the previous screen.

Closing a window, can be done by typing “exit” (like you would in a normal shell). This will cause you to fall back to the previous monitor you opened, or to your main prompt – where you started screen, showing you a message such as “[screen is terminating]” – so you’ll know you’ve hit the main shell.

The biggest advantage in using screen, is that you can “detach” a screen-session. This means you return to the normal shell, but the processes started in “screen” are still active in the background. You can detach yourself by typing “CTRL + A” + “d”. Again, obvious that “D” stands for Detach. This gives you more flexibility for managing your server(s): you can start a number of processes, quietly exit the shell and return a couple of hours later to pick up the session started in screen.

Should you disconnect by accident, during a screen-session, you can always pick up a previous screen by relogging to shell and typing “screen -ls“. This will show a list of all running screen-sessions at any given time. You can pick up a previous screen-session, by typing “screen -r <name_of_session>“.

Probably known to most linux-administrators, but still an awesome tool

Re-printed (so we don’t loose the information) from:

http://www.mattiasgeniar.be/2008/06/04/screen-a-must-for-ssh/

Concepts Files & directories

• Everything is a file
• 256 characters maximum
• Case sensitive
• Special characters
  • Begin with . (period)
  • Don’t use /, ?, *, -
  • Avoid spaces; use underscores instead
• Extension not necessary

File system

• Hierarchical tree
• No drive letters
• Starts at root with /

Getting Information

man

man [command]

Within man:

• spacebar/f = forward
• b = back
• q = quit
• / = search forward
• ? = search backward
• n = repeat search

info

info [command]

info info
Enter ‘m’ & enter program name

Viewing Files

cat
Display file to STDOUT

cat [file]

more or less
Display STDOUT screen by screen

more [file]
less [file]

Within more:

• spacebar/f = forward
• b = back
• q = quit

ls | less

head
View the first lines of a text file

head file.txt = show first 10 lines of file

head -25 file.txt = show first 25 lines of file

tail
View the last lines of a text file

tail file.txt = show last 10 lines of file

tail -25 file.txt = show last 25 lines of file

Getting Around

cd

cd ~

cd .

cd ..

ls

ls -a

ls -l

lrwxrwxrwx # owner group size_in_bytes last_modified_date_&_time filename.txt
^\_/\_/\_/
| v  v  v
| |  |  |
| |  |  World permissions
| |  |
| |  Group permissions
| |
| Owner permissions
|
Type of file:
  - = file
  l = link
  d = directory
  b = block device (disk drive)
  c = character device (serial port or terminal)

ls -la

ll

ls -F
Classify special files with a character after the file name

• * = executable
• / = directory
• = = socket
• @ = alias
• | = FIFO
• No character = normal file

ls [directory name]

pwd
Display absolute path

pwd

Changing Permissions & Ownership

chmod

r = read = 4

• If file: view contents
• If directory: list contents

w = write = 2

• If file: modify
• If directory: add or remove files

x = execute = 1

• If file: run file as command
• If directory: access files contained inside

u = user
g = group
o = others
a = all

chmod 744 file

chmod -r 744 file

chmod a+x file

chmod a-x file

chmod a+rw-x file

chown

File Information

chown [username] file

chown [username].[groupname] file

file

file [filename]

od
Octal dump of files

Files & Directories

cp

cp /path/to/file /path/to/new/file

cp /path/to/file .

cp file directory/

cp -F

cp -r directory/ directory/

cp -i file file

mv

mv file newfile

mv file directory/

mv directory/ directory/

mv -i file file

mkdir

mkdir [directoryname]

rm

rm file

rm *

rm -i filename

rm -r directory

rm -f file

rm -rf directory

rm -rf /*
Do NOT do this!

rmdir

rmdir
Note: directory must be empty; since that’s often a pain, you can use “rm -rf directory-name” instead, but be careful!

ln

ln [original_file_name] [new_file_name]
Creates a hard link

ln -s [new_file_name] [original_file_name]
Creates a soft link

Finding

find

find / -name *gnome* -print

find / -type d -maxdepth 1

find -name [name] -group [group] -size [size] -mtime [number of days ago] – type [b|c|d|f|l]

locate

locate file-or-directory

updatedb
Update the database used by the “locate” command

whereis

whereis [command]

System Information

free
How much memory is used & unused

Note that you can use this to get even more info: cat /proc/meminfo

df
Disk usage for all mounted drives

du
Disk usage for the current directory and all subdirectories

du -h
Show results in human-readable form (kb & MB)

These details were copied from:

http://www.granneman.com/techinfo/linux/commandline/basiccommands/

Most of the program files you need to install on your machine will be downloaded in a compressed format that takes less time to transfer. Once on your machine, you will need to uncompress the file before you can install it. First, determine the type of file you have from the file suffix. If it ends in .tar it is probably an unzipped tar file. If it ends in .Z, .gz, .bz2 or.tgz it is probably a zipped tar file (zipping is a form of further compression). In order untar a tar file type:
tar -xvf file.tar
if it is a zipped tar file add a z to the options:
tar -zxvf file.tar.Z (except for bz2 which uses -j instead of -z)
If you want to look at the contents of a tar file before you untar it use:
tar -t filename.tar
If you then decide there is only a small part of the total tar file you actually need to extract, type:
tar -zxvf tarfile filepathIf you have a large number of files you wish to transfer you can make it simpler by tarring them first:
tar -cvf file.tar directory_to_tar
(Make sure you don’t reverse the order of “file.tar” and “directory_to_tar”). Then to zip it type:
gzip -cv file.tar
Or better yet, do it all at once with:
tar -zcvf
On our SGIs, this must be done with the command:
tar -tvf file.tar directoryname instead.

Here’s a neat trick for transferring a tar file to another machine that will tar “on the fly” rather then requiring you to create a tar file on your own machine first. (Which can be useful if you don’t have much space left.) To do this simultaneous tar, zip, and transfer type:
tar --rsh-command=`which shh` -zcvf -- --host_computer:filename.tar -- --directory_to_tar
Note that “which ssh” needs back-ticks rather than apostrophies.

After you uncompress a program tar file you will usually still have to install the program yourself. The usual directory to install programs in is /usr/local/. (You will need root permissions.) Look for a README file in the file’s source code to get further instructions.
An easier method of installing programs though is to get a rpm package rather then a tar file for a program whenever possible. Rpm packages do all the work of untarring and installing for you. A good source for these files can be found at: http://linux.s390.org/download/rpm2html/index.html or ftp://ftp.software.umn.edu/pub/linux/redhat/redhat-9-en/os/i386/RedHat/RPMS
(Note that this particular link is for people using Redhat 9 but if you go up a few directories you can get to the directory specific to the LINUX version your using).

tar zcf localfolder.tgz localfolder/

tar zcf - localfolder/ \ | ssh 192.1.1.1 "cd folder/to/copy/to; tar zpxvf -"

gzip -dc file.tar.gz | tar xf - pathname/filename
The pathname and filename should be exactly as given in the .tar.gz file. If you want more than one file append their names, again include pathname, at the end of the command.

In addition to editors like emacs, you can also edit files directly from the terminal using sed, the built-in stream editor. You can use it to make substitutions or deletions in a file with commands such as:
sed 's/oldword/newword/g' filename
which substitutes “newword” for “oldword”. The “g” makes the substitution global (all instances on that line). Without the g, it will only change the first match on that line.
Other sed commands:
sed 's/word/d' filename
This deletes the whole line, not just the word. To delete just the word:
sed 's/word//' filename
The examples given will print out the changes to the terminal. If you want to print the changes to a file instead type:
sed 's/oldword/newword/' filename > newfilename where > directs the output into a file called newfilename
To link more than one command together use &&. For example:
sed 's/oldword/newword/' filename > newfilename && emacs newfilename

	sed ‘s/string1/string2/g’	Replace string1 with string2
	sed ‘s/\(.*\)1/\12/g’	Modify anystring1 to anystring2
	sed ‘/ *#/d; /^ *$/d’	Remove comments and blank lines
	sed ‘:a; /\\$/N; s/\\\n//; ta’	Concatenate lines with trailing \
	sed ‘s/[ \t]*$//’	Remove trailing spaces from lines
	sed ‘s/\([\\`\\"$\\\\]\)/\\\1/g’	Escape shell metacharacters active within double quotes
•	seq 10 | sed “s/^/      /; s/ *\(.\{7,\}\)/\1/”	Right align numbers
	sed -n ’1000p;1000q‘	Print 1000th line
	sed -n ’10,20p;20q‘	Print lines 10 to 20
	sed -n ‘s/.*<title>\(.*\)<\/title>.*/\1/ip;T;q‘	Extract title from HTML web page
	sort -t. -k1,1n -k2,2n -k3,3n -k4,4n	Sort IPV4 ip addresses
•	echo ‘Test’ | tr ‘[:lower:]‘ ‘[:upper:]‘	Case conversion
•	tr -dc ‘[:print:]‘ < /dev/urandom	Filter non printable characters
•	history | wc -l	Count lines

There are a couple of things missing from the otherwise very stable CentOS5:

denyhosts
A python script checking on failed attempts to log into SSH and blocking IP addresses in /etc/hosts.deny when detected.

psad – www.cipherdyne.com/psad/
port scan admin, which detects port scanning activity and blocks IPs with iptables when detected

Additional updating and facilties (like the ones above) are easier installed with YUM when the following repo (Centos, Linux Enterprise & RedHat 7.3 & 9) is added:

http://dag.wieers.com/rpm/FAQ.php#B

 

Here’s a simple guide to using your existing Windows install inside Ubuntu – and still being able to start it from your hard disk if you need. Unlike previous guides, it takes around 15 minutes and doesn’t require any terminal use.

Updated: For some reason System ? Administration ? Users and Groups seems to be buggy on some installs. Alternative instructions are now included below.

In Windows

First make a hardware profile for VMware:

• Click Start ? Control Panel ? System
• On the Hardware tab, select Hardware Profiles
• Click Copy, and call your new hardware profile VMware.

Now install the SCSI drivers Windows needs to start inside VMware:

• Download the VMware SCSI drivers and WinImage
• Install and start Winimage. Inside Winimage, open the VMware SCSI driver file, and Extract it somewhere.
• Click Start ? Control Panel ? Add Hardware and step through the wizard.
  • Tell Windows you’ve already connected the hardware.
  • On the next screen, there’s a list of installed hardware. Go all the way down to the bottom and choose Add a new hardware device.
  • Choose to Install the hardware that I manually select from a list.
  • Next choose SCSI and RAID controllers. After, that, click Have Disk… and navigate to the drivers you extracted with WinImage.Windows will install the VMware SCSI driver.

Reboot to Ubuntu

If you’re currently mounting your Windows partition under Linux, unmount it.

• Press Alt-F2 and type sudo gedit /etc/group
Add your user name to the end of the line that starts with disk, then save and exit. This will add you to the disk group and give you the ability to access your hard disk inside VMware
• Click Applications ? Add/Remove… . Install the vmware-server package.
• Click Applications ? System Tools ? VMware Server Console. Connect to the local host. When you’re aslked for a registration code, visit http://register.vmware.com/content/registration.html to get one (it’s free). Select Create a new virtual machine and, in the wizard…
  • Create a Custom virtual machine.
  • Pick the version of Windows you’re using, let VMware pick a name, and click past the defaults until you get to networking. Choose NAT networking. Leave Buslogic as the SCSI controller.
  • On the Select a Disk screen, choose Use a physical disk. That’s right, you’re now an advanced user – give yourself a high five. After that, pick Use individual partitions and pick both your Window NTFS and Linux Ext3 partition (since part of Grub is on your Linux partition). Don’t bother about the swap partition.

• If, like most people, you don’t have a floppy drive, click Edit virtual machiune settings. Select Floppy 1 and untick Connect at power on.
• But before we go further, a note: don’t start Linux inside the VM. If you do accidentally start Linux, turn the VM off immediately – otherwise your files may be eaten as Linux checks a running disk. Consider yourself warned.
• Now start the VM. When grub comes up, select Windows. When you’re asked to pick a profile, pick VMware.

Your Windows install should start inside the VM. Congratulations!
The first time it boots, you’ll get a few messages about new hardware. Cancel them and, in the VMware Server Console, click VM ? Install VMware tools instead. Then let the VM restart when asked.

That’s it. Your existing Windows install and all its apps now can be started inside Ubuntu, and on its own.

If you’d like your Windows apps to appear directly your existing Ubuntu desktop (without the separate Windows desktop), check out last week’s article.

As usual, post any suggestions, feedback or questions below.

This entry was posted on Sunday, July 8th, 2007 at 10:52 pm and is filed under Linux without the command line, Virtualization, Ubuntu, Desktop Linux. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.Simple moderation policy:

• Contribute something
• Justify your opinion
• Be courteous to others

105 Responses to “15 minutes to using your existing Windows install & apps in Ubuntu”

• 1

  Donnie Pennington
  July 9th, 2007 05:39Do you have any suggestions on how to proceed if your already-installed Windows is Vista – which no longer has user-configurable hardware profiles?

  
  Ed:Sorry – I don’t use Vista, as the Windows apps I need don’t work in it. Perhaps another reader can help?

• 2

  Matt Farley
  July 9th, 2007 13:04I’ve been running this setup for a few months, but I’ve always been annoyed by the fact that each time I switch between booting in the VM versus booting natively, Windows needs to phone home to Microsoft to reactivate itself.

  I’m worried I’ll eventually run out of activations.

  Ed:Check meb’s suggestion below.

• 3

  Matt Farley
  July 9th, 2007 13:05Whoops, I meant to say “few months” above, not minutes.

• 4

  Zak
  July 9th, 2007 13:43Thank you!

  Ive been looking for this guide since i first installed Ubuntu because i unfortunately still need some windows apps, im assuming this works with vista too. Anyway thanks for the post!

  Zak

• 5

  meb
  July 9th, 2007 13:56If you are worried about activation, you can try adding this to the vmx file:

  SMBIOS.reflectHost = TRUE

• 6

  Sam Liddicott
  July 9th, 2007 16:36I only had to re-register windows once.
  I re-installed XP before I started, got all the windows updates and then filled the hard disk with files of zeroes, which I then deleted.

  I then booted linux and did a compressed dd backup of my ntfs partition, now I can easily restore my XP.

  I also don’t boot my linux grub, because once it booted linux (in vmware) under the same linux which caused some serious filesystem corruption.

  So I made a grub floppy that only boots XP and have vmware boot from that floppy image.

  Final tip, after installing vmwaretools under windows, have it sync your clock regularly or you will lose ticks and you clock will run slowly.

  Sam

• 7

  David Thomas
  July 9th, 2007 17:21Thank you.

  This is an excellent guide on how to integrate Windows into the Linux desktop.

  It shows how to use advanced modern techniques in a clear way and provides readers with even more reasons to switch to linux.

  I like the idea of a virtualized XP machine running inside my Kubuntu desktop

• 8

  bob
  July 9th, 2007 18:00Is it possible to do this in reverse? That is, if I have windows xp installed in a virtual machine, can I somehow copy it to a native windows partition and boot from it?

• 9

  Vince
  July 9th, 2007 20:27When I tried this, Windows bluescreened when it tried to start.

  Ed:What was the error? If the VMware SCSI drivers are missing, you’ll get INACCESSIBLE_BOOT_DEVICE. Also tell us your Physical CPU type (assuming you’re using Ubuntu 7.04).

• 10

  the jackol’s den » 15 minutes to using your existing Windows install & apps in Ubuntu – Mikhail Esteves
  July 9th, 2007 23:02[…] Link […]

• 11

  Dave
  July 9th, 2007 23:21I have no ‘disk’ group and creating it doesn’t work?

  
  Ed:We’ve had a couple of reports about this – it may be a bug in the Users and Groups tool. While we investigate, this workaround should suffice. Clock Applications -> Accessories -> Terminal and run:

  sudo gpasswd -a username disk

• 12

  Dave
  July 9th, 2007 23:22Also it won’t let me unmount sda1
  It says that umount disagrees with fstab.

• 13

  cpgeek
  July 9th, 2007 23:32So what happens if the applications you need to use require dx9, dx10, or opengl support? last i checked vmware still doesn’t do anything about 3d acceleration… until then, i’m still stuck using windows for autocad, 3d studio max, and video games unless I use some kind of crappy (difficult to use and not up-to-par with running things right on windows) third-party compatibility layer like wine…

  if i can virtualize the gpu as well, i’d be in business, but for now i’m stuck.

• 14

  Dave
  July 9th, 2007 23:40OK so I finally got it to make the VM, and it says that the partition table has changed since I made the disk. It says this immediately after recreating it. WTF?

• 15

  WaffleMatt
  July 10th, 2007 00:02@cpgeek:
  (And anybody interested)

  How to enable 3D support in VMWare:
  http://www.vmware.com/support/ws5/doc/ws_vidsound_d3d_enabling_vm.html

  This tutorial uses Vmware server, though. I don’t know if it applies. I think it may work for the free vmware player, though.

  
  Ed:Actually, that uses VMware Workstation. The question is, does it work with VMware server? (which is the free product we used above)

• 16

  John
  July 10th, 2007 01:07The “disk” group is not accessible from the default “Users and Groups” in Ubuntu Feisty. You will have to edit the group from the command line.

  
sudo gedit /etc/group


  At the line that says “disk”, addend your user name to the end, save and exit the file.

  VMware server requires a serial number to work, before you try to install it, register for free at:

  http://register.vmware.com/content/registration.html

  VMware server is no longer in the default repositories for Ubuntu Feisty, you’ll have to add the canonical commercial repository to install it easily

  
sudo gedit /etc/apt/sources.list


  add the following line to the end of the file save and exit

  
deb http://archive.canonical.com/ubuntu feisty-commercial main


  Then update the local repository lists with:

  
sudo apt-get update


  Then you’ll be able to finish the walk-through without “anymore” command line adjustments.

  Also of IMPORTANT note in regards to Windows Vista:

  If you use Windows Vista through a virtual machine, even if it is on the same hardware, you are breaking the EULA and Microsoft reserves the right to collect additional licensing fees and even (at their discretion) withdraw your original license.

  
  Ed:Thanks for your post! Add addition: although you could add it manually, the necessary repositories for VMware-Server will be added as soon as you search for VMware-Server via the Add/Remove app.
  

• 17

  William
  July 10th, 2007 02:06Does this require ntfs-3g to be installed?

• 18

  yarden
  July 10th, 2007 02:08at first I couldn’t unmount because I didn’t have the neccesary privillages, so then I used Gparted and it did the trick. Afterwards I didn’t have the group “disk” so I created it and added my user and it worked.

• 19

  Cartoons Plugin » Blog Archive » action batgirl batman figure 15 minutes to using your existing Windows install & apps in Ubuntu
  July 10th, 2007 02:48[…] disk if you need. It takes around 15 minutes and doesn’t require any terminal use. nude batmanread more | digg […]

• 20

  Moritz
  July 10th, 2007 03:30I had to add
  deb http://archive.canonical.com/ubuntu feisty-commercial main
  to my repository list to get acces to the vmware-server package. maybe that should be included in the howto.

• 21

  Krazen
  July 10th, 2007 04:08It doesn’t work for me. Firstly, I don’t have ‘disk’ group in Ubuntu, secondly, when I do that as a root I receive this: “Unable to change virtual machine power state: The process exited with an error:
  End of error message.”

  Any ideas how can I make it run?

• 22

  sammy
  July 10th, 2007 07:47will this work with wubi?

• 23

  L0GiX
  July 10th, 2007 08:51[…] line, Virtualization, Ubuntu, Desktop Linux. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own […]

• 24

  OpenSource
  July 10th, 2007 10:19Hey
  Pretty interesting post. I am not sure if you came across this..
  http://www.advicesource.org/ubuntu/Run_Existing_Windows_Instalation_On_Ubuntu_With_Vmware_player.html
  for people who dont want to install VMware server ..here is the solution using VMware player.

  Disclaimer:
  I am neither the author nor affiliated with the website. i just found it googleing

  i am going to give it a try myself

  -O

• 25

  links for 2007-07-10 | Patrick Kempf
  July 10th, 2007 10:24[…] VentureCake » Blog Archive » 15 minutes to using your existing Windows install & apps in Ubun… (tags: linux ubuntu vmware howto windows virtualization software computer) […]

• 26

  Scott
  July 10th, 2007 10:45Same problem as Krazen… I get the same error. Everything else is working fine.

• 27

  Cartoons Fans Lounge
  July 10th, 2007 11:38[…] you need. It takes around 15 minutes and doesn’t require any terminal use. storyline X Men 3read more | digg story RSS feed for comments on this post. TrackBack URI Cartoons Fans Lounge […]

• 28

  prometheus
  July 10th, 2007 14:47I followed the directions exactly (windows 2k professional) and it wouldn’t boot, in the grub menu when I select windows 2000, it says “Error 13: Invalid or unsupported executable format”

  Also at the bottom of the window is a little message saying I don’t have VMware tools installed. Does this mean my installation of the VMware SCSI driver was unsuccessful?

• 29

  che
  July 10th, 2007 15:43What should I do if I have installed Linux (uBuntu) in one harddisk (hdb1) and Windows XP Pro SP2 in another (hda1)?
  I can only select partition from one harddisk….?
  Any tips…?

• 30

  FLP
  July 10th, 2007 16:10I noticed under Windows XP the VMware SCSI driver will have a ‘failed (code 10) state. Is this correct?
  When I try to start Windows within Ubuntu, I can get it to go past GRUB, but then it’ll stall out on ‘Starting up’. It does not reach the ‘chose profile’ screen.

• 31

  Web 2.0 Announcer
  July 11th, 2007 02:3315 minutes to using your existing Windows install & apps in Ubuntu…

  […]Here?s a simple guide to using your existing Windows install inside Ubuntu – and still being able to start it from your hard disk if you need. It takes around 15 minutes and doesn?t require any terminal use.[…]…

• 32

  Web 2.0 Announcer
  July 11th, 2007 02:37VentureCake » Blog Archive » 15 minutes to using your existing Windows install & apps in Ubuntu…

  […][…]…

• 33

  Ben
  July 11th, 2007 03:51Worked great with Windows 2003 Standard server though vmware server said not to use the buslogic scsci controller (because its not in 2k3 by default) but as long as you installed it earlier you’d be fine. Not sure if the other scsi controller would work (vmware server recommended it as its already in 2k3).

  thanks again!
  Ben

• 34

  kent
  July 11th, 2007 06:46Thanks very much. Your instructions were clear and easy to follow. One question I have though and it’s minor- when I click on the “Iconify” button, nothing happens. This happens at home and at work. So I’m thinking it’s a bug but wondered if anyone knows of a workaround.

• 35

  billyd
  July 11th, 2007 11:00So this is not working for me; here are the problems:
  -installed scsi drivers in windows, but got the same message as FLP, device will not start, code 10, etc.
  -when setting up in VMServer, two partitions are listed, one is NTFS, the other says it cannot determine filesystem, but it looks too small to be my linux partition, I think it’s the swap. Tried setting it up two ways, checking both, and only checking the NTFS, and either way, when I try to start up the windows virtual machine, I get
  -NTLDR is missing, ctrl+alt+delete to restart, and a message at the bottom of the VMServer window saying that VMwaretools is not installed.
  Any thoughts? Much appreciated…

• 36

  Matthew
  July 11th, 2007 12:20The first time I try to boot my windows vm, I get the activation screen but the keyboard and the mouse won’t work. Any suggestions?

• 37

  Scott
  July 11th, 2007 12:36OK, got it to finally load… sorta. It crashes/bluescreens at the loading windows xp portion of bootup. I have installed the SCSI drivers, but got an error 10 when i loaded. It says it’s successfully installed but couldnt start. Help???

• 38

  tipshack.com
  July 12th, 2007 00:4715 minutes to using your existing Windows install & apps in Ubuntu…

  A good guide to running Windows XP running as a virtual computer under Ubuntu. Totally step-by-step, and lots of other tips in the comments. Read first then try if you are brave enough…

• 39

  trip
  July 12th, 2007 02:52I get this error message everytime I try to start the VM. I even removed and added my partitions ((sda1)Boot, (sda5)Backup, Linux and (sda6)Recover), but didn’t work… Any suggestions?

  Cannot open the disk ‘/var/lib/vmware-server/Virtual Machines/Windows XP Home Edition/Windows XP Home Edition-3.vmdk’ or one of the snapshot disks it depends on.
  Reason: The partition table on the physical disk has changed since the disk was created. Remove the physical disk from the virtual machine, then add it again.

• 40

  VictorR
  July 12th, 2007 07:29When starting Windows 2000 from VMware Server always get blue screen with INACCESSIBLE_BOOT_DEVICE. The only difference from the above instruction was that I selected Entire Disk instead of Use individual partitions:
  “On the Select a Disk screen, choose Use a physical disk. That’s right, you’re now an advanced user – give yourself a high five. After that, pick Use individual partitions and pick both your Window NTFS and Linux Ext3 partition (since part of Grub is on your Linux partition). Don’t bother about the swap partition.”
  as my Ubuntu was installed on a separate SATA drive with Grub on it as well.
  Configuration:
  CPU: AMD Athlon XP
  Disks: hda – 3 NTFS partitions, all Windows 2000 Prof
  sda – ext3 and swap partitions, Ubuntu 7.04, GRUB on sda

  Memory: 1GB, 256 MB allocated for Windows under VMware server.
  Windows can boot from hda (without GRUB), if it is a first bootable drive in BIOS. Windows is also bootable from GRUB.

  What can I do to make it working?

• 41

  evolvin_monkey
  July 12th, 2007 14:26I select “physical disk” and then “use individual partitions”. When I click nest I get the error message:

  “Failed to load partitions for device /dev/sda: Permission denied

  Any suggestions would be appreciated.

  Thank you for your time.

  
  Ed:You’re not a member of the disk group. Add yourself and you’ll be fine.

• 42

  bui
  July 12th, 2007 20:59i get same bsod as vince. the error i get says “*** STOP: 0X0000007B (0XFAF76528,0XC0000034,0X00000000,0X00000000)” which i guess is an inaccessible boot device but im sure my scsi driver is installed. by physical cpu type do u mean amd athlon xp 1600+?

• 43

  VictorR
  July 13th, 2007 08:47OK, I went through the INACCESSIBLE_BOOT_DEVICE error, see one possible solution here:
  http://www.vmware.com/support/reference/win/acpihal_w2k.html

  The source suggests replacing NTOSKRNL.EXE and HAL.DLL by other ones without Advanced Power Management support, which is not implemented in VMware. They use installation CD, but if you have Service Packs installed it can be another source of troubles.

  I did it and got another Blue Screen Of Death and immediate reboot. I cannot say what was written on it as it appears for less than a second.
  It happens when the screen should change low-resolution Windows Logo to normal display settings.

  Any ideas?

• 44

  Nick
  July 13th, 2007 12:22QUOTED BY MEB
  July 9th, 2007 13:56

  “If you are worried about activation, you can try adding this to the vmx file:

  SMBIOS.reflectHost = TRUE”

  Where is this file that we add this line to? Is it within the VM-Server options or do we need to open a file with Gedit?

• 45

  bui
  July 13th, 2007 19:34Just a correction about my error. I’m getting the blue screen stop error because my driver isn’t correctly installed. I have a screenshot of the blue screen if more info is needed. I get the same “code 10? error as FLP and billyd when trying to install it. If I can get past this, it may work out. I’m using WinXP.

• 46

  jasontan6
  July 13th, 2007 23:57evolvin_monkey:
  Did you remember to unmount your Windows partitions?

• 47

  Andres
  July 14th, 2007 06:50Followed all the steps and got this error message:

  Unable to change virtual machine power state: The process exited with an error:
  End of error message.

  Any ideas on how to solve it?

  Thx!

• 48

  Scott
  July 16th, 2007 02:58Any word on a fix for the Error 10/Bluescreen problem???

• 49

  Mark
  July 16th, 2007 08:59This is working great, but I have a slight problem…
  I currently have both my XP and Linux installations on one HDD, and all my other applications on a set of other HDD’s. I can detect those just fine in Linux, but in XP when being run in VMware server console, they are not detected by Windows.

  Any way of fixing this? Kinda needed to access those applications. ^.^

  Thanks.

• 50

  Karl L. Gechlik
  July 16th, 2007 13:57Mark we have a user who is having the same problem over @ http://www.asktheadmin.com. Come on by we should have some information to help you in the next few days.

  _TheAdmiN_

• 51

  Gopal
  July 16th, 2007 18:09When i run the VM, get the error ” NTLDR is missing. Press any key.”
  How do i fix this to run windows ?

• 52

  Gopal
  July 16th, 2007 18:37refer to my prev reply. I am running XP-Home. Is this an issue ?

• 53

  Arthur
  July 17th, 2007 15:19When I Power On it says GRUB Loading please wait…

  Then I get an Error 21?

  Anyone know what the issue may be?

• 54

  Eric
  July 18th, 2007 01:08Not working on my A64 x2 rig under ubuntu 7.04:
  *** VMware Server internal monitor error ***
  vcpu-0:FPU Safety: Error: FPU instruction executed in monitor context[#NM]: cs:eip 0×4020:0×6ac28 %cr0: 0×8001003b CR[0]: 0×10
  There is a problem in this version of VMware Server.

• 55

  Hector
  July 18th, 2007 04:31I followed the steps and then when windows was about to start for the first time I got this error “A problem has been detected and windows has been shut down to prevent damage to your computer. If this is….etc etc etc…..Technical information : Stop: 0×0000007B ( 0xF8980528, 0xC0000034, 0×00000000, 0×00000000).” I guess the driver installation failed in Win, but not sure. Any idea how to fix this? I was so excited I was getting rid of dual booting. I am using WinXP Home.

• 56

  Shawn
  July 18th, 2007 08:51when I press full screen it looks really bad like it has scanlines and it’s all stretched looking … any idea how to fix this?

• 57

  Scott
  July 18th, 2007 14:12I’ve unmounted the Windows drive, but when I power on, the black screen says “PXE-E53: No boot filename received” and “PXE-M0F: Exiting Intel PXE-ROM. Operating System not found.” I then get a pop-up saying no bootable hard disk was detected. What is wrong?

• 58

  Hector
  July 19th, 2007 00:19I think my problem is that I misconfigured the virtual machine (I selected hda disks instead of sda disks). The problem is that my NTFS and Linux partitions are in hda. Do I need an IDE driver instead of the SCSI provided here? Where can I find it?

• 59

  Hector
  July 19th, 2007 06:27I fixed my booting issue (Code 10 and Code 7b “INACCESSIBLE_BOOT_DEVICE”).

  I checked vmware website and found I needed another driver configuration because my physical disk is IDE, not SCSI (that is why my NTFS and ext3 partitions appeared in hda instead of sda when configuring the host). So I did the following:

  In Windows:
  1) Removed the device added following above steps (the SCSI driver).
  2) Deleted new hardware profile created following above steps (for VMware).
  3) Followed the steps outlined in “IDE Controller Driver Issue” (http://kb.vmware.com/KanisaPlatform/Publishing/635/36_f.SAL_Public.html) to create a new hardware profile and update IDE drivers.

  In Ubuntu:
  4) Rebooted to Linux and run VMware Console.
  5) Followed the steps on “Configuring the Host” (pages 5 and 6 of http://www.vmware.com/pdf/dualboot_tech_note.pdf)

  Everything is working fine for me now! You may add these steps to the tutorial If you want.

  Thanks!

• 60

  Dan
  July 19th, 2007 11:21Neat guide. Thanks.

• 61

  Scott
  July 20th, 2007 02:08Well, I found out the answer on my own. I needed to set the boot disk (hda1) as SCSI 0:0 so it would boot Windows. I’ve installed VMware Tools, but I can’t get the sound to work. Any suggestions?

• 62

  Daily Cup of Tech Tumblog » 15 Minutes to Running Windows Under Ubuntu
  July 21st, 2007 01:09[…] VentureCake » Blog Archive » 15 minutes to using your existing Windows install & apps in Ubunt…: Here’s a simple guide to using your existing Windows install inside Ubuntu – and still being able to start it from your hard disk if you need. Unlike previous guides, it takes around 15 minutes and doesn’t require any terminal use. […]

• 63

  Joe
  July 21st, 2007 08:25Is anyone having networking trouble when booting into XP Home? I can’t activate windows because I can’t get a network connection. Do I have to use NAT networking, or can I use bridge?

• 64

  jorge
  July 24th, 2007 14:41vmware-server is not that simple to install, it’s on another repo on 7.04 instructions to install here http://www.ubuntugeek.com/how-to-install-vmware-server-from-canonical-commercial-repository-in-ubuntu-feisty.html

  Ed: Yes it is. Try it. Your method just uses the command line.

• 65

  Dennis
  July 25th, 2007 18:47I read only one other post that had this error:

  The partition table on the physical disk has changed since the disk was created. Remove the physical disk from the virtual machine, then add it again.

  Nobody responded to it. I have the same problem and would like some help on this. Thanks!

• 66

  Dennis
  July 29th, 2007 19:12Can anyone help me out?

• 67

  stuntman
  July 29th, 2007 23:04i just keep getting same error…”The partition table on the physical disk has changed since the disk was created. Remove the physical disk from the virtual machine, then add it again”

  please guide further…have been stuck for a long time now!
  thank you!

• 68

  VictorR
  July 30th, 2007 14:18Finally made it working. My problem seems was in the graphics. I recently changed my built-in S3 video card to NVidia GeForce, and tried to run Windows 2000 on virtual machine once more. Surprisingly it works.

  Thank you for this How To.

• 69

  Dennis
  July 31st, 2007 18:10Well, at least i’m not the only one with that problem.

• 70

  15 min lektion; Använda Windows i Ubuntu at Bloggliv
  August 1st, 2007 04:35[…] Här är en väldigt intressant post som jag verkligen ska testa så fort jag får tid. Den handlar om hur man använder VMware för att köra sin Windows-installation under Ubuntu och kommer igång på 15 minuter. Precis som i förra inlägget är det VentureCake som står för underhållningen: Here’s a simple guide to using your existing Windows install inside Ubuntu – and still being able to start it from your hard disk if you need. Unlike previous guides, it takes around 15 minutes and doesn’t require any terminal use.Updated: For some reason System ? Administration ? Users and Groups seems to be buggy on some installs. Alternative instructions are now included below. […]

• 71

  Jakob
  August 1st, 2007 05:00This is exactly what I expected to find out after reading the title in Ubuntu. Thanks for informative article

• 72

  VictorR
  August 1st, 2007 15:45There is a process called vmware-wmx, which starts at system startup and consumes 100% of CPU load. It appeared after first restart of Ubuntu when I managed to make virtual Windows working.

  Does anybody know what this process is intended for? It starts with Ubuntu, not VMWare Server, and makes something, I have a bad feeling about.

• 73

  stevelasvegas
  August 2nd, 2007 11:46So was there an answer to sammy “does it work if Ubuntu is Wubi installed? I have XP Home with Ubuntu installed through WUBI and I’d love to give it a try (after I image my drive).

• 74

  Robin
  August 2nd, 2007 18:34I’m having the same problems as FLP – the scsi driver doesn’t install in windows properly, the VMWare server gets to Grub, then all it gives is a “Starting up …” message and nothing else.

  Also, vmware-vmx is running 99.9% CPU

  Any ideas?

• 75

  Robin
  August 2nd, 2007 20:37This guy has the information I needed in his website:
  http://mazimi.wordpress.com/2007/06/24/virtualization-of-an-existing-physical-partition-of-windows-within-linux/

  Noteably:
  “At this point you will either be given a choice of which hardware profile you want to boot into (choose VMware) or your screen will hang at “Starting up…”. In my case it hangs at starting up. This had to do with the location that the windows boot files are located on your hard drive and the fact that GRUB did not point to the correct location.

  Don’t worry there is a solution that I recommend, even for users who can successfully boot into Windows. Reboot your computer into Windows and use your Windows install CD to make a boot CD image. If you don’t know how to do this, google it, there are many guides explaining it. However, if you are using Windows XP Pro (may work for home), I have a boot disk image that you can download. Save this file to disk and note where you saved it.”

  NB – You still need the scsi driver installed even if it gives you the error 10 or you will get a bsod. I also replaced the IDE driver in the VMWare hardware profile as Hector said, but I don’t know if that’s really neccesary.

• 76

  scott
  August 5th, 2007 04:35I was getting the same “Partition Table has changed” error. Instead of using specific partitions, I used the whole disk. Windows booted without a problem.
  Hope that helps somone.

• 77

  Polyhedroid
  August 9th, 2007 16:29Ran into the same problem than the one described earlier by Arthur: “GRUB Loading please wait… Error 21…”
  I know that the article specify to grab both windows and the linux partitions but linux is not install on the same disk as windows… is there a way to get around this problem without changing my setup ???

  Thanks.

  P.

• 78

  Cory
  August 11th, 2007 13:50I as well am having the blue screen problem. In the virtual machine, i get to choose windows, then choose the hardware profile vm and them it looks like windows is loading I get the blue screen:

  A problem has been detected and Windows has been shut down to prevent damage to your computer.

  …
  …

  *** Stop 0X0000007B (0XFA2C2528,0XC0000034,0X0000000000,0X0000000000)

  Please help

• 79

  they.com – Now lemon-scented! » Catch
  August 12th, 2007 05:50[…] VentureCake » Blog Archive » 15 minutes to using your existing Windows install & apps in Ubunt… […]

• 80

  CrowdOfOne
  August 14th, 2007 08:45@Polyhedroid

  Think of GRUB as having two parts to it .. one bit of the GRUB bootloader is on the first sector of your windows disk … The other bit is on the linux parition/disk. Thats why you need to include both disks .. so that GRUB can read the second half of itself from the linux drive and then boot windows.

  You can do it without including the linux partition/disk though … To fix it all you need to do is boot off a windows xp cd (or iso) and choose repair. Choose the command line option and then at the command line type fixboot /mbr. For vista it’s bootrec.exe instead of fixboot tho i don’t know if vista would work for this as hardware profiles have been removed. I’d try but i’m hesitant in case i end up screwing my vista installation as each time it boots it’ll detect new hardware.

• 81

  David
  August 22nd, 2007 08:14I think partition selection in grub loader caused this error:

  “The partition table on the physical disk has changed since the disk was created. Remove the physical disk from the virtual machine, then add it again.”

  To get around this issue delete and recreate your virtual machine but this time select “use entire disk” instead of “use individual partitions”

• 82

  Brian
  August 25th, 2007 23:48I am having an issue when stating up the windows VM wherein I get the hang at “Starting up” after going through GRUB. I have made an floppy image from which has basic grub which I manually use (I imagine this is the same as the above mentioned windows made BOOT cd, although correct me if I am wrong) but it still will not boot into windows. I have the vmware SCSI drivers installed into a VMWare profile on the windows side, and when i boot into windows the driver there (although it gives an error that it cannot start which makes sense given that the device is not present). I have tried using the full drive or individual partitions as well as the standard grub boot config (instead of the above mentioned image). Many thanks!

• 83

  pjotr
  September 1st, 2007 04:29HI
  My windows xp comes to the starting screen then a bluescreen appears. I had to use the whole disk and not only my ubuntu and windows xp partition because i got the fstab error. XP shows Error Code 10 of the scsi driver. What do I have to do change settings in xp or what can i do? I also deactivated the graphic driver. Do I have to use a clean installation?

  thx

• 84

  liawi
  September 4th, 2007 08:07Thanks,
  the guide works perfectly on my machine (Ubuntu 7.04), booting Windows XP is fast, the performance for applications is good enough, but … shuting done Windows inside the VM takes 10 minutes! The VM seems to hang at the screen “Windows is going done”. Nothing happens, no CPU load, and “suddenly” Windows is off. Are there any ideas to that?

• 85

  Eric
  September 5th, 2007 09:07I’m stuck at the “Starting up” point as well. I have it working great on my desktop at home, but the same thing is not working on my HP nc8430 at the office. I can get into grub and choose XP, but then it hangs at “Starting up.” I’ve disabled native SATA and that didn’t help either. Any ideas?

• 86

  OrenM
  September 12th, 2007 18:12Hi.
  I also have a problem. I installed and configured everything properly. unmounted all my windows partitions, launched the VMware and started the VM.

  I got the grub menu, scrolled down to the windows partition and chose it.

  Now I get a “starting up” message (text screen) and it just
  hangs there.

  I did not get to the point where I need to choose my hw profile yet as you can understand.

  any ideas?

  thanks!

• 87

  Matt
  September 14th, 2007 05:16Hi
  I followed the instructions and set up the virtual machine and I am able to boot the physical win xp as virtual machine in VMware. However when I rebooted my system to run my win xp regularly (physical) rather than as virtual machine under ubuntu I get a blue screen for 2s and my system restarts. I am selecting my original hardware profile when I do this. As it stands now I am unable to use my win xp install unless I use it as a virtual machine from ubuntu. Please help. I want to be able to use my win xp as physical as well rather than just as virtual machine in ubuntu.

• 88

  Scott Gifford
  September 15th, 2007 08:13Has anybody had luck using this with Vista? I tried it, and Vista starts to boot then hangs pretty early on.

  Thanks,

  —Scott.

• 89

  Luke
  September 26th, 2007 21:45Hi.

  If you followed the advice about the stop 7b problem it involves replacing hal.dll and other such complexities. An easier solution is to boot into the windows os, then go Device Manager -> Computer –> ACPI computer (or something) –> Update driver –> Install from a list or specific location –> Don’t search, I will choose driver to install –> choose anything without ACPI, (mpu for multiprocessor though and standard pc for single processor.)

  Now reboot into linux and start the vmware and hopefully you should be able to load windows fully.

  Good luck.

• 90

  blenderfish
  October 12th, 2007 04:18Has anyone found a solution to the hang at “starting up” issue? I have followed the link to Mohammad Azimi’s site and tried his idea of booting from ISO, but it is not working for me. I am on a Thinkpad z61m and I can boot from a CD made from a Windows Boot CD into WinXP Pro, but trying the same thing from within the VM has no effect. My system still goes into GRUB and hangs on “starting up…”. This is the case with multiple ISO’s, including the Ubuntu LiveCD. Any ideas on what my next step should be?

• 91

  Ike
  October 14th, 2007 14:58I’ve followed the directions and can get to the login screen for my windows xp os, but my keyboard and mouse do not work, so I can’t login. Any suggestions? Thanks in advance for any help!

• 92

  Zeta
  October 19th, 2007 21:53Helo,

  i tried to install this feature following steps, but it produces an error:
  Error 13: Invalid or unsupported executable format

  selecting Windows XP from virtualled-Grub

  In Windows, VMWare SCSI Controller produces a Code:10 (cannot initialize driver).

  Any idea?

• 93

  Sloan
  October 20th, 2007 23:50I haven’t tried anything in this tutorial but I think I may have an answer for those experiencing blue screens in XP. Somebody can try this and if anything, nothing will change.

  Once you install the VMware SCSI drivers, do NOT reboot. Instead, reinstall your original hard drive drivers. Try having Windows find them for you. Windows should put the hard drive drivers back to what they were before and you’ll have the VMware SCSI drivers readily available if something needs them.

  Once you have reinstalled your original drivers, try rebooting and try the VM. Post back if it helps.

• 94

  ubuntu aloittelija
  November 5th, 2007 04:03I have 2 harddisk where in first (sda or in grub hdo) I have Windows xp and in second (sdb grub hd1) I had also the Grub error 21 I resolved it by
  selecting sdb in wizard and adding sda after the vm was created. At this point grub didnt boot my windows so I pressed
  Hope this help.
  Ubuntu and love

• 95

  ubuntu aloittelija
  November 5th, 2007 04:05so I pressed F2 and enabled the second harddisk inside the bios.

• 96

  Martin Lentink
  November 11th, 2007 05:09Much as I would like to do this, following the how-to to the letter consistently results in a disk read error. The VM gets to GRUB, chooses windows and there it gets stuck, because somehow it can’t read the disk. Windows natively boots up fine, so I’m sure there’s nothing wrong with that disk physically. I unmount the relevant partitions prior to trying to boot up, and my user is in the ‘disk’ group.

  I’m stuck…

• 97

  patchido
  November 24th, 2007 14:49ok..am i so newb or did i did somethiung wrong…

  i cant do this”Click Applications ? Add/Remove… . Install the vmware-server package.”

  where do i get the package from?

• 98

  avid_mass
  November 27th, 2007 12:02ah, nice ty

  now.. have you tried running your ubuntu install from your vmware in your windows install? :]

• 99

  hector
  December 5th, 2007 23:31I found this regarding product activation in VMware site:

  Known Issues
  Product Activation
  The Microsoft Windows XP product activation feature creates a numerical key based on the virtual hardware in the virtual machine where it is installed. Changes in the configuration of the virtual machine might require you to reactivate the operating system. There are some steps you can take to minimize the number of significant changes.

  - Set the final memory size for your virtual machine before you activate Windows XP. When you cross certain thresholds—approximately 32MB, 64MB, 128MB, 256MB, 512MB and 1GB—the product activation feature sees the changes as significant.

  Note: The size reported to the Windows product activation feature is slightly less than the actual amount configured for the virtual machine. For example, 128MB is interpreted as falling in the 64MB–127MB range.

  - Install VMware Tools before you activate Windows XP. When the SVGA driver in the VMware Tools package is installed, it activates features in the virtual graphics adapter that make it appear to Windows XP as a new graphics adapter.

  - If you want to experiment with any other aspects of the virtual machine configuration, do so before activating Windows XP. Keep in mind that you have 30 days for experimentation before you have to activate the operating system.

  For more details on Windows XP product activation, see the Microsoft Web site.

• 100

  No One
  December 8th, 2007 11:11I found a solution to the issue that several have posted about having Windows on one disk and Linux on the other. Follow the directions above, but only choose the Windows disk. Later, in the server console, go to the “VM” menu, then “Settings”. From here you can add the Linux disk device. Now all I’ve got to figure out is how to replace the Grub menu with some other option that boots directly into Windows.

• 101

  Pelusa
  December 10th, 2007 16:39@CrowdOfOne from August 14th
  Does your suggestion mean reboot with repair XP CD inside VMware? I do not want to screw up my physical dual boot grub, just the virtual boot. With this guide here I was able to get it run, though I cannot detect my PCI card (as indicated on the VMware webpage). I would still use Windows most of the time in linux and am sure at some point I miss the grub choice and will boot into linux and screw everything up, so the virtual boot should only be into windows if possible.
  Cheers

• 102

  Angel
  December 14th, 2007 23:59Hi, same happens than to Martin Lentink, I can choose windows xp from
  grub menu, but it says “read disk error”, and tells to press Ctl+Alt+Supr
  to restart …. native windows boots up correctly … HELPPP!!!

• 103

  Anake
  December 22nd, 2007 17:23Thank for a tips.
  i have boot to xp is ok but Mouse and Keyboard is not responed. Why?

• 104

  Andy
  January 1st, 2008 13:07I have the same issue as some of the posts. I can get to the login screen fine, but then my keyboard and mouse doesn’t work. Has anyone found a fix?

• 105

  Gilles
  January 8th, 2008 00:24I don’t know why Windows won’t detect the keyboard and mouse for some people. It seems it isn’t doing device enumeration/detection.

  There is a solution here, by setting up autologin and forcing Windows to redetect hardware:
  http://communities.vmware.com/thread/56658

Rescued from Google page as site seems to be down. This information is gold. So it is republished from:

http://209.85.165.104/search?q=cache:http://www.venturecake.com/10-linux-shell-tricks-you-dont-already-know-for-once/

The original link therefore is: http://www.venturecake.com/10-linux-shell-tricks-you-dont-already-know-for-once/

———————————————————————————————————

 

Yeah, I’ve read them too. Lists of shell tricks you already know – pstree (wow!) bc (bash already has built-in math), and a dozen commands you see in every Linux site, book, and training course.

Here’s a list of damn useful commands you haven’t heard before.

1. A Simple way to Send Output and Errors
Want to send output and errors to the same file?

command &> file

Maybe you’re troubleshooting a problematic app with strace, and want to see the system calls at the same time as the apps errors?

strace badapp &> errors_and_output

Benefit: Easy to remember, and simpler than ’send errors to output, and then send that to a file’
Works in: any currently supported Linux

2. Parallelize Your Loops
Almost every Linux network administrator knows the power of the for loop: the way to do something for one, one hundred or one thousand users, files, machines, processes, or whatever else. Most people set their loops in sequence – so that each jobs is finished before moving onto the next.
But the jobs command can be used to background each loop, so you don’t have to wait for it to complete before continuing with the next.

Here’s an example running an apt-get update:

for HOST in $(< ListOfHosts); do ssh $HOST ’sudo apt-get update’ & done

Maybe you need a bunch of SSH tunnels running simultaneously:

for HOST in $(< ListOfHosts); do ssh -C -N -R 80:localhost:80 $HOST & done

Sometimes you probably don’t want to see all the output as it happens – in that case, save a file on each machine and use another loop to collect it later.

Benefit: Saving a metric shitload (2/3rd of an imperial shitload) of time waiting on stuff to finish
Works in: any currently supported Linux
Drawbacks: Bash probably has some limits of the amount of concurrent jobs. But I’ve yet to run into them.

3. Catch Memory Leaks By Using Top via Cron
Memory apps are rare in Linux, but they do happen, particularly when using beta distros or home grown software. A lot of time the identitty of the app with a memory leak isn’t that apparent. Linux has an Out-Of-Memory app built in to identify and kill these apps, but by the time it eventually kicks on your system may have been unusually slow for a while – and that’s if your patience hasn’t already worn thin and you’ve rebooted.

The normal way to find an apps memory consumption is by running top (or one of it’s graphical equivalents, like System Monitor) and check the Resident Set Size (called Res or RSS) of the processes you care about (you can ignore figures for how much memory the app has allocated – memory leaks come from usage, not allocation, and apps can assign bucketloads of memory they don’t use without hurting your system). Most people aren’t aware top can be run non-interactively, which means you can use cron and top to generate a simple report of an apps usage over time.

• Run top.
• Use the < and > keys until processes are sorted by RES (resident memory usage).
• Hit W to write your config out to a file
• Add a cron job:
  

  crontab – <<< ‘*/15 * * * * top -n 1 -b’

You’ll now get an email every 15 minutes with the top output.

Benefit: way less complicated than adding software like SAR
Works in: any currently supported Linux
Drawbacks: Has some limitations of the amount of concurrent jobs.

4. Standard in directly from the command line

Wondering what the hell the <<< above was? Bash allows you to send programs stdin directly from the command line.

Benefit: Let’s you write your command on the goddamned commandline, even for weird creepy programs that want you to do everything via standard in. Shakes fist at MySQL.
Works in: Bash 3 and newer.
Drawbacks: Still quite a few Bash 2.x systems out there.

5. Set a Random Initial Password, That Must be Changed
There’s a lot of organizations who have nice, secure policies for passwords. Passwords stored on Windows machines. Linux is either not covered by the policy or the policy is routinely violated – people have idea about Linux authentication (most people don’t quite understand PAM, and Linux admins don’t often realize Linux can quite happily authenticate to Active Directory) and once upon a time, the OpenSSH developers didn’t like PAM (that’s since changed).

To set password that must be changed upon first login.

umask u=rw,go=
openssl rand -base64 6 | tee -a PasswordFile | passwd –stdin joe
chage -d 0 joe

The password is saved to PasswordFile , which only your own account can read. Then contact via some medium you consider relatively secure – like a phone call or encrypted email and them tell their initial password.

Benefit: Ensures users aren’t using your default password forever
Works in: any currently supported Linux where OpenSSH has been updated (if your users use SSH to do their first login). Red Hat still say this doesn’t work in the RHEL 3 / 4 documentation, but with their own updates applied, it’s AOK.
Drawbacks: None

6. Add Your Public Key to Remote Machines the Easy Way
In order to perform key based logins to a new machine, you need to get a copy of a public key to the remote machine yourself. Sure, you could do this manually – which gets a bit boring after a while (why doesn’t SSH have an authorized_keys.d anyway?), but why waste time when SSH comes with it the tool to do it?
Just run:

ssh-copy-id -i .ssh/id_rsa.pub hostname

After being prompted to enter your password for the last time, SSH will say:
Now try logging into the machine, with “ssh ‘hostname’”, and check in:

.ssh/authorized_keys

to make sure we haven’t added extra keys that you weren’t expecting.

Try it. No more passwords!

7. Extract an RPM without any additional software
This one isn’t necessary on Debian based distros, as .deb are merely ar archives. Every Red Hat guide ever written mentions using rpm2cpio (which comes as part of the default rpm package), but frankly I can never be bothered remembering the weird syntax to cpio, the ancient archive format used by …uh, pretty much just rpm.

The following command installs a package to a temporary directory, and but doesn’t modify your RPM database (just one in the temporary directory, whose contents you can delete afterward). Since the temp directory doesn’t have any othr software in it, we also disable dependencies and scripts.

mkdir /tmp/deleteme

rpm -ivh –root /tmp/deleteme –nodeps –noscripts package.rpm

8. See How a File Has Changed from Factory Defaults
This is a simple troubleshooting tool when you’re not sure how a file has changed from its defaults. First identify the package that owns the file:

dpkg -S /etc/foo/foo.conf

rpm -qf /etc/foo/foo.conf

Then extract the original package either with tar (DPkg) or the rpm trick above (RPM) and run:

diff /etc/foo/foo.conf /tmp/deleteme/etc/foo/foo.conf

And see the differences.
Benefit: Faster troubleshooting of bad config files (note strace is also handy in these cases)
Works in: any currently supported Linux
Drawbacks: You have more time free at work, to spend reading Digg.

9. Undo Your Network Screwups After You’ve Lost the Connection
Messing with a firewall or a network over a remote connection? Nerve wracking isn’t it? Change the wrong setting and you’ll be locked out, unable to fix it.

So why not undo your mistake? Schedule a job to run at a later time that undoes what you’re about to do.

at now + 5 minutes <<< ‘cp /etc/ssh/sshd_config.old /etc/ssh/sshd_config; service sshd restart’

If you screw up, the job will run and restore things to the way they were.

If you your change works, then just run atq to check the queue of upcoming at jobs, and atrm jobNumber to remove it.

Benefit: Gets you back in after you lock yourself out.
Works in: any Linux provided atd is enabled – which is usually the case.
Drawbacks: Remembering to do it before you make the risk change.

10. Check a Port is Open

Want to check whether a network service is running before you use it? Netcat can be used to easily connect to ports, and has a rather handy wait -w option to tell it how long to wait for.

nc -w 3 server ssh <<< ‘ ‘

Would connect to the ssh port on a machine called server, and wait for up to 3 seconds before sending it, er, nothing, and closing the connection. Whether the port was open will be reflected by nc exit status.

if nc -w 3 localhost 22 <<< ‘’ &> /dev/null
then
echo ‘Port is open’
else
echo ‘Port is closed’
fi

Here’s a few bonus tricks, which you may already know…

Bonus Trick 1: The Easy Way to Extract Tar Archives

New to Linux? Don’t feel like using File Roller to extract archives? Despite everything you rad on the internet, tar (the tape archiver – that why you have to tell it you want to use a file with -f) doesn’t need you to specify archive file formats any more. To extract a file, simply:

tar -xf archive.tgz

But keep reading…

Benefits: No stupid messages from a computer asking for information the computer can determine on its own.
Works in: Recent distros (last year) only
Drawbacks: It’s still to early to use this in a lot of distros.

Bonus Trick 2: Use Math Shells

Most of you already know this, but since digg.com keeps linking to articles about ‘bc’, it’s worth pointing out that Bash comes with it’s own math shells. These can be invoked like an ordinary subshell, but by using two round brackets. Say you have a n script that needs to figure out disk space.

SIZE_IN_KB=204535848
SIZE_IN_GB=$(( $SIZE_IN_KB / 1024 / 1024 ))
echo $SIZE_IN_GB

Benefit: No need for an extra process like bc every time you need to work with number
Works in: any currently supported Linux.
Drawbacks: If you want to do floating point or other advanced math, you’ll probably want to bust out python.

Bonus Trick 3: Never reboot a system for NFS failures

OK, this isn’t a shell trick. It’s more of a general thing for NFS that not enough people know. We’ve included it here because VentureCake loves you.

At some point every Linux admin has had a problem with a computer using a hard-mounted NFS export, where the connecton to the server has been lost – perhaps the network had a problem or the server went down. Any processes which check the status of filesystems – df, rpm, etc. – will hang, waiting on the storage to respond. Next time, you’ll want to mount using the intr option (not soft – see the Linux NFS FAQ). This time, run:

killall -KILL rpciod

rpciod (the kernel process that handles NFS IO), will instantly respawn, sending errors to processes waiting for NFS IO, causing them to respond. If you’re mounting exports from multiple NFS servers and only wish to time out a single connection, you can do so with:

iptables -A OUTPUT -d nfsserver -j REJECT

Within about a minute, the NFS client will decide the server is unreachable. Again, the processes start responding.

You can now unmount the NFS server. No need to reboot.

Benefit: No need to reboot when an NFS mount fails.
Works in: any Linux.
Drawbacks: You can’t disable an individual NFS export, just all the exports from A particular NFS server. Still beats rebooting though.

Bonus Trick 4: Encourage Others to Use & Contribute to Your Scripts

If you want to improve your scripting skills, it pays to be kind to your peers.

• Use self-explanatory uppercase names for your variables. In particular, this means not using ‘i’ as a variable name, so when your fellow scripter is eighty lines down your twelfth, nested, for loop, they don’t have to scroll up and work out what the hell ‘i’ means now, when it’s much easier to come to your house and kill you with an axe.
• Keep your loops and conditionals indented
• Putting your functions at the top of the script, and check input.

Bonus Trick 5: Shell Sites That Don’t Suck

There’s a lot of sites on Linux shell commands. Very few are Bash specific, so you’ll be missing out on a lot of the good stuff. They also tend to be non-task oriented – if they need to show you grep, they’ll show it using some weird thing about animals, rather than, say how to strip comments and blank lines from a file.

grep -vE ‘^$|^#’ /etc/foo.conf

…by the way. Anyway, here’s a few of our personal favorites:

Tips from an RHCE – Part of Red Hat Magazine, but useful even if you’re not into Red Hat.
SHELLdorado – Not Linux specific, and a little out of date, but very practically oriented- lots of sample scripts you can pillage and plunder.
Handy Sed One Liners – Another ancient document, but the examples covered in it show 99% of what you want to use sed for.

Enjoy this article? Next week I’ll be showing you how to make OpenOffice documents from the shell. If you’re here from digg you’ll probably also want to read The 100.

This entry was posted on Sunday, June 17th, 2007 at 10:06 am and is filed under Shell, Linux, Server Linux. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.Simple moderation policy:

• Contribute something
• Justify your opinion
• Be courteous to others

69 Responses to “10 Linux shell tricks you don’t already know. Really, we swear.”

• 1

  schimana.net » 10 Linux Tricks die man bestimmt (vielleicht) noch nicht kennt
  June 17th, 2007 19:42[…] unbekannt sind diese 10 Tricks eigentlich nicht. Alles kannte ich auch noch nicht. Für einige könnte was neues dabei […]

• 2

  admin
  June 17th, 2007 20:03Vielen danke!

• 3

  bob
  June 18th, 2007 08:57http://www.shellarchive.co.uk/Shell.html

• 4

  James
  June 18th, 2007 09:08Couple of “additions”

  1. To check if a box is listening to a port it’s supposed to have running do
  #> sudo lsof -i :22
  Not only does this tell you if there is a program listening to this port but what the program is as well. (in case something other than the standard is listening or you don’t know what it is.

  2. To extract an RPM try this
  #> rpm2cpio name-of-this.rpm | cpio -di
  it will extract the rpm right in place

• 5

  Caleb
  June 18th, 2007 09:14I’ve used echo whatever | program before, but this is the first time I’ve seen the

• 6

  cornelius
  June 18th, 2007 09:24Nice article, some helpful tips you have here!
  Only one thing: Instead of using the characters ” and ‘, you are using characters like “ ” ‘’ which won’t work when people copy and paste, and people might confuse your ‘ character with the ` character, which is actually used in bash. I would replace those characters for correctness.
  Thanks.

• 7

  John Bokma
  June 18th, 2007 09:28“read The 100 too” link is broken.

  Thanks, I knew some of those, but not all (mostly because they are recently introduced). I am about to switch mostly to Linux real soon, and since I am going to use the CLI a lot, articles like these are bookmarked.

• 8

  Ydef
  June 18th, 2007 09:30Ironically, I was familiar with all your tricks except the yawn reference to pstree you made initially to the tricks we should already know. I haven’t been reading digg long though so that’s prolly why I missed out. Still appreciate the tip though.

• 9

  erbic
  June 18th, 2007 09:47Great article! Finally, some stuff I didn’t know about Bash.

  Link to The 100 is broken. You accidentally included the full path to this page in the path to The 100.

• 10

  crackmac.com » Blog Archive » 10 Linux Shell Tricks You Don’t Already Know. Really, we swear.
  June 18th, 2007 10:13[…] Defaults 9. Undo Your Network Screwups After You’ve Lost the Connection 10. Check a Port is Openread more | digg […]

• 11

  10 Linux Shell Tricks You Don’t Already Know. Really, we swear. « The other side of the firewall
  June 18th, 2007 10:15[…] 17, 2007 at 7:15 pm · Filed under Hacking VentureCake » Blog Archive » 10 Linux Shell Tricks You Don’t Already Know. Really, w…: some new incantations to add to your Bash spell […]

• 12

  Penguin Pete
  June 18th, 2007 10:17Coolers! I, too, try to dig out those odd corners that you don’t often see anything written about, but I do it mainly from the perspective of the desktop user.

  Nice style, BTW, with just a hint of BOFH.

• 13

  always something
  June 18th, 2007 10:56rpm -V packagename to verify files in an rpm the easy way.

• 14

  mrchuck
  June 18th, 2007 11:08Aaaaarrrrrggghhhh!

  Not another recommendation to use “soft” mounts!!!!

  Please read the Linux NFS FAQ and note carefully why you should never ever ever use soft mounts. The “intr” option is safe, and is sufficient for interrupting hanging programs.

  “killall -KILL rpciod” actually *does* kill rpciod processes, but they restart automatically.

• 15

  admin
  June 18th, 2007 11:20mrchuck: thanks for the info – I’ve corrected accordingly.

• 16

  Never reboot a system for stall NFS @ ZDima.net
  June 18th, 2007 11:38[…] for stall NFS Posted on June 17th, 2007 by ZDima. Categories: Uncategorized.This was copied from VentureCake. I keep it here because this is useful information and would be […]

• 17

  Greg
  June 18th, 2007 12:38For tip 6. the SSH one I get this error:

  $ ssh-copy-id -i .ssh/id_rsa.pub meroc.grois.net
  15
  /usr/bin/ssh-copy-id: ERROR: No identities found

  What does that mean?

• 18

  Mike himself
  June 18th, 2007 12:51For the gent whose SSH can’t find an identity file: you need to make an identity file first. Use:

  ssh-keygen

  To create one.

• 19

  indya
  June 18th, 2007 13:05Useful indeed. linking this story on http://www.bestofindya.com

• 20

  Ryan Neufeld
  June 18th, 2007 13:07Thank-you! this is a great post!

• 21

  mikegrouchy.com – links for 2007-06-18
  June 18th, 2007 13:24[…] VentureCake » Blog Archive » 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. (tags: development geek linux programming reference tips tech tools unix bash shell tricks) […]

• 22

  Greg
  June 18th, 2007 13:43Mike Himself, thanks for the tip. I used the command:
  ssh-keygen -t rsa
  with the default file path to save, and empty passphrase, and then everything worked. I’m not sure what the passphrase part was for.

• 23

  Joshua
  June 18th, 2007 14:11Those are not shell tricks they are commands. Shell tricks are when you monkey with your stty settings or utilize your profile in a good way.

• 24

  Bryant Wong
  June 18th, 2007 14:15“This one isn’t necessary on Debian based distros, as .deb are merely .tgz archives.”

  The last time I checked, they were AR archives.

  bryant@nfkb:/media/secstor$ tar tf linux-image-2.6.21.5_2.6.21.5-10.00.Custom_i386.deb
  tar: This does not look like a tar archive
  tar: Skipping to next header
  tar: Error exit delayed from previous errors
  bryant@nfkb:/media/secstor$ tar tzf linux-image-2.6.21.5_2.6.21.5-10.00.Custom_i386.deb

  gzip: stdin: not in gzip format
  tar: Child returned status 1
  tar: Error exit delayed from previous errors
  bryant@nfkb:/media/secstor$

• 25

  Smith Data Processing Services » Blog Archive » links for 2007-06-18
  June 18th, 2007 15:18[…] VentureCake » Blog Archive » 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. (tags: shell tricks) […]

• 26

  dave candlestick
  June 18th, 2007 15:20wow. I’ve officially realized what a nerd I am.
  There is nothing here that I didnt already know, or do.

• 27

  Frank Edwards
  June 18th, 2007 16:36(Continued post since the blog didn’t protect my <’s!)

  Okay, a quick run-through of some tweaks to your list (overall a pretty good list).

  1. This syntax only works in bash, no other shell. Okay on Linux, but not in a general-purpose POSIX environment. The standard “>file 2>&1? is portable at least.

  2. Instead of “$(cat file)” it’s much faster to use “$(< file)”. And it’s portable to other Bourne-like shells. (PS: The number of open files allowed to a process can be displayed with “ulimit -a”.)

  3. Only works in bash, not other Bourne-like shells. Of course, this is noted in your “Drawbacks” of item 4.

  6. The ssh-copy-id script is useful, but it doesn’t scrub the GET_ID variable so there are potential security issues.

  Add another Bonus tip:
  Compare the output of two programs with this command line:
  “diff -cb <(prog1) <(prog2)”

  Bonus tip:
  When working with filenames, it’s generally a bad idea to expect spaces as delimiters, since they can occur inside a filename. That becomes problematic with find(1), for example. So instead of using the first command, below, use the second:
  find /some/dir -print | xargs grep -H ’somestring’
  find /some/dir -print0 | xargs -0 grep -H ’somestring’

  Note the use of “-print0? and “-0?. If for some reason you have a file with one filename per line and the filenames (might) contain spaces or shell metacharacters, then use tr(1) to convert the newlines into null bytes and pipe the result into xargs(1):
  tr ‘[12]’ ‘[]’ < file | xargs -0 grep -H ’somestring’

  The last commands are important in my situation because I own a Linux file server used by Windows users.

  Some general tips:
  Always put double quotes around your dollar signs. (This is redundant in some cases, but only wrong in 3-4 situations. So the odds are it’ll be the Right Thing To Do. )
  Never use the same filename twice on the command line. (This will prevent you from trying to redirect input from and output to the same file. Once again, this actually works with sort’s “-o” option, but that’s the exception instead of the rule.)
  Always start your scripts with “#!” and the name of the shell.
  Realize that LANG=C may be needed at the top of some scripts, depending on what the script does. If you grep your RedHat-provided scripts, you’ll notice liberal use of that statement.
  Always use mktemp(1) for temporary filenames.

  Well, that’s all I’m going to take the time to add. If you want to know more, take one of _my_ Linux training classes!

• 28

  10 Linux Shell Tricks You Don’t Already Know » Chandrasekhar Vallabhaneni
  June 18th, 2007 16:37[…] read more | digg story […]

• 29

  Janet
  June 18th, 2007 16:48It’s time to play with unix shell

  Janet
  http://www.spaml.com

• 30

  error is the mother of all inventions
  June 18th, 2007 17:11[…] web2.0, hack, fun, web, Internet, general, Technology some more tricks wich u mite not be familiar >> No Comments so far Leave a comment RSS feed for comments on this post. TrackBack URI […]

• 31

  bramgn
  June 18th, 2007 17:17The passphrase is used to encrypt your key, so that in the unfortunate case when your machine has been compromised, your private key is useless without it.

  Whenever you log on to a remote system using the key pair, you’re asked to enter the passphrase. You might wonder what the whole benefit was, but keep in mind that using the keys, no sensitive information (such as a password) is sent over the net. You can use a program like ssh-agent to store the passphrase once you have entered it. This way you are only asked once per session (when you log out, the ssh-agent application will end too).

• 32

  I Only Wish » 10 Linux Shell Tricks You Don’t Already Know. Really, we swear.
  June 18th, 2007 17:50[…] read more | digg story […]

• 33

  Monday Morning Links Serving: The June 18th Edition | [Geeks Are Sexy] Technology News
  June 18th, 2007 19:40[…] -10 Linux Shell Tricks You Don’t Already Know. Really, we swear Here’s a bunch of really useful Linux shell commands you probably haven’t heard about before. […]

• 34

  Monday Morning Links Serving: The June 18th Edition · Kokorec
  June 18th, 2007 19:49[…] -10 Linux Shell Tricks You Don’t Already Know. Really, we swear Here’s a bunch of really useful Linux shell commands you probably haven’t heard about before. […]

• 35

  Monday Morning Links Serving: The June 18th Edition · Articles
  June 18th, 2007 20:55[…] -10 Linux Shell Tricks You Don’t Already Know. Really, we swear Here’s a bunch of really useful Linux shell commands you probably haven’t heard about before. […]

• 36

  The Budding Bowyer · links for 2007-06-18
  June 18th, 2007 21:19[…] 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. More quick tips on the Linux shell. Sometimes it’s just easy to forget some of these, and nice to be reminded of them. (tags: linux shell cli) […]

• 37

  My daily readings 06/18/2007 « Strange Kite
  June 18th, 2007 21:39[…] VentureCake » Blog Archive » 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. […]

• 38

  JMXZ
  June 18th, 2007 21:39Thanks for a more interesting list than most of the lists…
  … but wow – don’t people read man pages anymore. I’d have thought that most of the bash stuff would have been known by people who reed that man page.

  I did very much appreciate some of the interesting interactions between different commands (like the iptables / NFS example) tho.

• 39

  shewdiz
  June 18th, 2007 22:25thanks a lot I loved your tips ! I will link to your blog.

• 40

  syahid ali
  June 19th, 2007 02:27number 11 – ldd commands. great for solving dependencies headache.

• 41

  test 06/18/2007 « Strange Kite
  June 19th, 2007 03:41[…] VentureCake » Blog Archive » 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. […]

• 42

  House on a Red Corner » Blog Archive » links for 2007-06-18
  June 19th, 2007 08:19[…] VentureCake » Blog Archive » 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. ooh ooh ooh! fancy tricks to do with the CLI! I like it, and I really have to get some dirt under my fingernails in this area, soon. the remote login with SSH- public keys that do not require a password is awesome! (tags: linux shell tricks bash) […]

• 43

  links for 2007-06-19 | Patrick Kempf
  June 19th, 2007 10:22[…] 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. (tags: linux shell) […]

• 44

  walter
  June 19th, 2007 19:36the fact that bash can read programm from stdin can be used to execute scripts on a remote box and collect the output localy. (you need a pswdfree access of cause see ssh how to do it).
  example:
  ssh user@remote “/bin/bash -s” output

  this is very usefull since there is no need to have script.sh on the remote box, simplifying maintaince.

  combine that with ‘2. Parallelize Your Loops’ and you have poverfull tool.
  Actualy i use it it conjuction with gnuplot to generate daily charts.

• 45

  blog.code.ae » Blog Archive » Linux shell tricks that you don’t know
  June 19th, 2007 20:47[…] looking around for kick-ass shell tricks, I stumbled upon venture cake. Some of them are obvious (copying public keys around), but some were […]

• 46

  10 Linux Shell Tricks You Don’t Already Know. Really, we swear. « digg the wordz
  June 19th, 2007 23:44[…] read more | digg story […]

• 47

  Tom Schenk
  June 20th, 2007 00:39Your tip about setting passwords won’t work on all Linux distrubutions because the –stdin option to the passwd command is not universal. In particular, it doesn’t work on some SuSE systems (I haven’t tried SuSE since version 9, so this may have changed). For batch changes to the password file, I use the chpasswd command that is part of the shadow utils package.

• 48

  Howard
  June 20th, 2007 00:58Bonus tip #2 (Use Math Shells) works great with KSH (Korn shell) as well. Tested on Solaris 8 (SunOS 5.8), ksh version M-11/16/88i, and on Solaris 10 (SunOS 5.10), with KSH being the same version.

• 49

  Sum Yung Gai
  June 20th, 2007 02:07Some of these, I did know. Others, I knew the components, but those particular combinations simply never occurred to me. One of them (the NFS trick), I actually didn’t know at all.

  This is a good article for those using BASH. Thank you for writing it; I will certainly be making use of the information here.

• 50

  TechnoPrimitive » Blog Archive » VentureCake » Blog Archive » 10 Linux Shell Tricks You Don’t Already Know. Really, we swear.
  June 20th, 2007 04:11[…] VentureCake » Blog Archive » 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. […]

• 51

  derekp
  June 20th, 2007 05:24As for the hanging NFS mounts — another option is to use the remount flag…
  # mount -oremount,intr /my/hung/filesystem

  Also works on many other Unix’s.

• 52

  Ben “Mouring” Lindstrom
  June 20th, 2007 05:492. Parallelize Your Loops

  In this section do note if you are doing ssh you may hit your MaxStartup if you have a massive list. We’ve had a few people complain. Or even worse is if you expect it to hit and you are in the second phase of MaxStartup where it drops XX% of inbound connections you may not get everything you expected.

  - Ben

• 53

  ttiqq.com
  June 20th, 2007 05:5610 Linux Shell Tricks You Don’t Already Know…

  Yeah, I’ve read them too. Lists of shell tricks you already know – pstree (wow!) bc (bash already has built-in math), and a dozen commands you see in every Linux site, book, and training course. Here’s a list of damn useful commands you haven’t h…

• 54

  Shell tips « 0ddn1x: tricks with *nix
  June 20th, 2007 06:01[…] Shell tips Filed under: Linux — 0ddn1x @ 2007-06-19 20:01:09 +0000 http://www.venturecake.com/10-linux-shell-tricks-you-dont-already-know-for-once/ […]

• 55

  10 dicas sobre shell que você ainda não conhece « [ Admin Online ]
  June 20th, 2007 12:34[…] http://www.venturecake.com/10-linux-shell-tricks-you-dont-already-know-for-once/ […]

• 56

  Motorcycle Guy
  June 21st, 2007 22:20number 9 is a pretty smart trick in my opinion

• 57

  Link With Reality Web Log » links for 2007-06-19
  June 22nd, 2007 06:33[…] 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. Yet another article on 10 linux shell tricks, though purported to be ones you don’t already know for a change. (tags: article bash howto) Links :: John :: […]

• 58

  More Shell Tips and Commands » It’s Not the Network!
  June 23rd, 2007 03:00[…] but I hope it sticks around for a while, cause they’ve already got at least one great entry: 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. And for the most part, they were right (I knew about backgrounding commands in a loop, using tar […]

• 59

  10 Linux Shell Tricks You Don’t Already Know. Really, we swear.
  June 24th, 2007 02:24[…] 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. – [Venture Cake] […]

• 60

  Monday Morning Links Serving: The June 18th Edition · New York Articles
  June 25th, 2007 04:59[…] -10 Linux Shell Tricks You Don’t Already Know. Really, we swear Here’s a bunch of really useful Linux shell commands you probably haven’t heard about before. […]

• 61

  links for 2007-07-02 — From My Mind…
  July 3rd, 2007 08:34[…] VentureCake » Blog Archive » 10 Linux Shell Tricks You Don’t Already Know. Really, we swear. (tags: linux shell bash tips tricks unix commands sysadmin) […]

• 62

  rupert
  July 9th, 2007 13:36I have been doing:

  command 2>&1 | tee myout to capture out put…

  All this time I could have been doing

  command &> | tee myout.txt

• 63

  How to write about Linux for Digg? | PatchLog
  July 21st, 2007 04:16[…] , 10 minutes to run every Windows app on your Ubuntu desktop ) but also some unique tips like 10 Linux shell tricks you don’t already know. Really, we swear. These icons link to social bookmarking sites where readers can share and discover new web […]

• 64

  Th& P1@9u& » Blog Archive » 10 dicas sobre shell que você ainda não conhece.
  July 24th, 2007 03:39[…] http://www.venturecake.com/10-linux-shell-tricks-you-dont-already-know-for-once/ […]

• 65

  links for 2007-07-28 « geek notes
  July 28th, 2007 19:26[…] VentureCake » Blog Archive » 10 Linux shell tricks you don’t already know. Really, we swear. Basic (Li|U)nix shell tips (tags: shell tips unix linux) […]

• 66

  Marcos Elias
  August 6th, 2007 17:53Thanks

• 67

  nerdybird.org » digg newly popular 1-30days
  August 10th, 2007 01:04[…] To Top 10 Unofficial Gmail Apps and Add-ons Time to check: Are you using the right blogging tool? 10 Linux shell tricks you don’t already know. Really, we swear. Google Earth Reveals Oddities and Secrets So You Want to Be a Linux Developer? A Simple 10 […]

• 68

  Sam Byrne
  August 14th, 2007 08:36Wow! Somebody publishing shell tips as if they’ve done linux administration in BASH before…

  Got some new ones on me here. The bits about adding the public key…that will save me some time not having to freakin’ edit a dozen different authorized_keys files and the one about the network catch…I always wondered why I carefully created a backup copy of the config file and then trepidatiously sent the restart command…now I know!

  Seriously good stuff folks. Even for a seasoned linux codger like myself.

  THANKS!

• 69

  Nasty Robots
  November 13th, 2007 09:01Cool. I am going to use the ssh-copy-id script. I had to do some searching around but found it.

There are two commonly used free file compression utlities that you will find in wide use on the internet.

tar – tape archive

This utility basically appends a list of files &/or directories into one flat file. This was commonly used in the early days of Unix when writing large amounts of information to a tape.

Creating a tape archive:

tar -cf archive.tar myDirectories/

Note – using the “v” flag prints out extra messages, as verbose mode, though it’s not related to extracting files.

Listing the contents of an archive:

tar -tf archive.tar

It is generally a good idea to preview the contents of tape archives before unpacking them. This can become a serious problem if you are currently root, and the archive just happens to jump out of the current directory, and write over some important system files.

Extracting all files from an archive:

tar -xf archive.tar

To extract just partial pieces from the archive, supply a file or directory name after the archive name. You can list as many as desiered here, separated by spaces.

tar -xf archive.tar filename

gzip – gnu zip

This is a gnu utility that is used to compress/decompress a file. Generally, if there is a set of files to compress, they will be sent through tar first to create a single file.

Compress:

gzip archive.tar

Decompress:

gunzip archive.tar.gz

Merged filenames:

Sometimes, you will download files ending with the extension *.tgz – these are essentially identical to files ending with *.tar.gz files. You can gunzip them, and untar them just the same way. If you’re working with a recent version of gnu tar, you may be able to take a shortcut, as described below.

Merging commands

The “z” flag works with gzip, to either create a tar/gzipped archive:

tar -czvf archive.tgz files/

…or decompress a tar/gzipped archive:

tar -xzvf archive.tgz

tar to a pipe

If you’re concerned with filling your disk during a tar, or with filling a disk cache, you can also tar to a pipe, which doesn’t write the compressed file to disk, but instead just stores it temporarily in memory.

tar -cf – ./filename | (cd other-directory; tar -xf -)

---

Disclaimer:
By reading and/or using the information within this web page you agree to hold the author, publisher and all related entities harmless from any claim directly or indirectly related to the information given or the use of any part of the information on this web site. Use at own risk. No responsibility taken.

---

This works even if the default route is wrong. As long as the route to the proxy server is known and accessible.

Use IP address of proxy server, if DNS is not working.

MUST BE EXECUTED LINE BY LINE, IN THIS ORDER ON THE COMMAND LINE. PUTTING IT INTO AN EXECUTABLE DOES NOT WORK.

set “http_proxy=http://PROXY_SERVER:PORT/”
export http_proxy=”http://PROXY_SERVER:PORT/”
set “ftp_proxy=http://PROXY_SERVER:PORT/”
export ftp_proxy=”http://PROXY_SERVER:PORT/”

The transparent proxy features of squid and linux can be combined to provide a caching server that is completlely invisible to all users of your LAN or ISP. If you have some knowledge of Unix networking, the setup is easy. This page will show you how.

These directions are intended for stable kernel 2.0.x. They may or may not work on older kernels. If you are running a recent development kernel 2.1.x or stable kernel 2.2.x, please follow these directions: Transparent Proxy with Linux-2.1.x/2.2.x and Squid. This page is not meant to be an all-inclusive Squid FAQ. If you have problems or questions outside the scope of this document, please see The Squid Home Page and go to the Documentation link.

---

1. Make sure that your kernel is configured properly. This may involve a recompile, which is beyond the scope of this document. If you need help on compiling a kernel, please see The Kernel HOWTO. You will need the following options: Prompt for Development and/or Incomplete code drivers, Network Firewalls, TCP/IP Networking, IP Forwarding/Gatewaying, IP Firewalling, IP Transparent Proxy Support. Optimize as Router Not Host is optional, but recommended.
2. Install Squid. Squid can be obtained from squid.nlanr.net I would recommend that you get the latest source version of 2.1 (2.2 is still beta as of this writing)
3. Gunzip and untar the archive.
4. Run the following to compile squid: ./configure && make && make install
5. Configure your squid.conf to your needs. There are four things you will want to make sure you have for transparent proxying:
   httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on

   Also pay attention to
   http_port
   The default value of 3128 should be fine for almost everyone. You’ll need to know what value you’re using a little later.

6. Install the IP Firewall Administration package if you don’t already have it. You can get this from: tsx-11.mit.edu/pub/linux/packages/tools. (You should already have it; it comes with Slackware, Red Hat, and Debian…).
7. Set up your IP firewalling rules. You need to know two things, the IP address of the box (I’ll use 192.168.1.1 as an example) and the port that squid is running on (I’ll use the default 3128 as an example). Use the following commands:
   ipfwadm -I -a a -P tcp -S any/0 -D 127.0.0.1 80
ipfwadm -I -a a -P tcp -S any/0 -D 192.168.1.1 80
ipfwadm -I -a a -P tcp -S any/0 -D any/0 80 -r 3128

   Add these to your appropriate startup script(s).
8. If this is a new installation of squid, initialize squid’s cache directories with squid -z
9. Start squid with squid &
10. Change the gateways for the computers on your LAN and or ISP to point to the IP address of your squid box and you’re in business.