As a Linux user, I’ve used

# mount -o loop -t iso9660 /path/to/file.iso /mnt/tmp

more times than I can count. Not suprisingly, you can do it in Solaris 10 as well, there’s just another step involved:

# lofiadm -a /path/to/file.iso /dev/lofi/1
# mount -o ro -F hsfs /dev/lofi/1 /mnt/tmp

The first command, lofiadm, associates the iso file to a block device managed by the kernel LOopback FIle driver. The second command is the same old mount command you’re used to, you just point it to the lofi device. To unmount:

# umount /mnt/tmp # lofiadm -d /dev/lofi/1

Running GUI’s over an SSH tunnel is nothing new, and not too tricky. However, I recently had a case where I wanted to run a java-based GUI (dhcpmgr) as root on a remote host, and have it display on my local Linux box. I hit a couple of snags, so I thought I’d post them here to hopefully help out others. First, we need to configure the Solaris 10 SSH server to allow remote SSH forwarding. First, make sure you have the following in your /etc/ssh/sshd_config file:

X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes

If you had to make changes to the file, restart SSH by doing a

svcadm refresh ssh

Now, onto the Linux client. If X11 forwarding in general is working, but you get a blank window when running Java apps, here’s what you need to do. Edit /etc/ssh/ssh_config and make sure the following is in there:

ForwardX11Trusted yes

Finally, we have both ends setup, now it’s time to make the connection. From the Linux client:

ssh -CX mynonrootuser@solarisbox

The -C option compresses the connection, the -X option sets up X11 forwarding. Now, we need to run our app as root. Don’t do like your fingers want you to do here! My fingers are so trained to typing ‘su -’ to become root, but the trailing dash blows away some important environment variables. Instead, do a

su

without the trailing dash! Now, fire up your Java GUI, and bookmark this page. You’re a Unix SysAdmin, and you need GUI’s so little that you’ll surely forget how to do all this when you need it next ;-)

Here’s a step-by-step howto on how to configure the ALOM card on a Sun Fire server so that it’s accessible over the network via SSH. It should also work for Sun Fire V210, V215, V240, V245, V445, V250, V440, and V445 servers as well. You must first program the card via the operating system. You can do this via the serial cable or SSH session.

• First, cd into the proper directory:

  cd /usr/platform/uname -i/sbin

• Now, show the current version of firmware on the card:

  ./scadm version

If it’s not version 1.6, download the latest from Sun

* Current latest version of firmware is 1.6.9, you will download the file ALOM_1.6.9_fw_hw0.tar.gz
* Run the following commands to update firmware:


mkdir /usr/platform/`uname -i`/lib/images
cd /usr/platform/`uname -i`/lib/images
gzip -dc ~/ALOM_1.6.9_fw_hw0.tar.gz | tar xvf - 
/usr/platform/`uname -i`/sbin/scadm download boot alombootfw; echo "done, now sleeping for 60 seconds"; sleep 60
/usr/platform/`uname -i`/sbin/scadm download alommainfw

After a couple minutes, the card should be ready for use.

• Now display the current configuration like so:

  ./scadm show

You should get output like this:

if_network="true"
if_modem="false"
if_emailalerts="false"
sys_autorestart="xir"
sys_bootrestart="none"
sys_bootfailrecovery="none"
sys_maxbootfail="3"
sys_xirtimeout="900"
sys_boottimeout="120"
sys_wdttimeout="60"
netsc_tpelinktest="true"
netsc_dhcp="false"
netsc_ipaddr="0.0.0.0"
netsc_ipnetmask="255.255.255.0"
netsc_ipgateway="0.0.0.0"
mgt_mailhost=""
mgt_mailalert=""
sc_customerinfo=""
sc_escapechars="#."
sc_powerondelay="false"
sc_powerstatememory="false"
sc_clipasswdecho="true"
sc_cliprompt="sc"
sc_clitimeout="0"
sc_clieventlevel="2"
sc_backupuserdata="true"
sys_eventlevel="2"

• Let’s setup the card so that it can be reached via SSH. Run the following commands, replacing the IP settings with your own:

  ./scadm set if_network true ./scadm set netsc_tpelinktest true ./scadm set netsc_ipaddr 192.168.1.18 ./scadm set netsc_ipnetmask 255.255.255.0 ./scadm set netsc_ipgateway 192.168.1.1 ./scadm set sc_cliprompt srv1-mgmt ./scadm set mgt_mailalert “me@my.com 3” ./scadm set mgt_mailhost 192.168.1.5 ./scadm userpassword admin ./scadm set if_connection ssh ./scadm set if_emailalerts true

• Verify your configuration by doing a

  ./scadm show

When done, you need to reset the SC, like this:

./scadm resetrsc

You’re done!

I recently had to setup the RSC card on some v490’s to enable remote console access. Unfortunately, these cards only do telnet, and not SSH, but when you need console access, you have to have it! If nothing else, at least SSH to a server sitting on the same subnet as the RSC card, then telnet over from there, limiting your exposure. Follow the jump for a step by step howto on how to configure the RSC.

• You must first program the card via the operating system. You can do this via the serial cable or SSH session. Begin by installing the latest SUNWrsc package.
• After installing the package, you need to run the configuration wizard:

cd /usr/platform/SUNW,Sun-Fire-V490/rsc/
./rsc-config

The following is the output from the rsc-config wizard, adapt it to your own settings:

Continue with RSC setup (y|n): y

Set RSC date/time now (y|n|?) [y]: y
Server Hostname [bkeapp1]: 
Edit customer info field (y|n|?) [n]: 
Enable RSC Ethernet Interface (y|n|s|?) [n]: y
   RSC IP Mode (config|dhcp|?) [dhcp]: config
   RSC IP Address []: 192.168.1.16
   RSC IP Netmask [255.255.255.0]: 
   RSC IP Gateway []: 192.168.1.1
Enable RSC Alerts (y|n|s|?) [n]: y
   Enable Email Alerts (y|n) [n]: y
      SMTP Server IP address []: 192.168.1.5
      Setup Backup SMTP Server (y|n) [n]: n
      Email address []: me@my.com
Enable RSC Serial Port Interface (y|n|s|?) [n]: y
   Serial port baud rate (9600|19200|38400|57600|115200) [9600]: 
   Serial port data bits (7|8) [8]: 
   Serial port parity (even|odd|none) [none]: 
   Serial port stop bits (1|2) [1]: 
Setup RSC User Account (y|n|?) [y]: y
   Username []: admin
   User Permissions (c,u,a,r|none|?) [cuar]: 



Verifying Selections


General Setup
  Set RSC date now  = y
  Server Hostname   = serverhostname
  Set Customer Info = n

  Is this correct (y|n): 



Ethernet Setup
  IP Mode      = config
  IP Address   = 192.168.1.16
  IP Netmask   = 255.255.255.0
  IP Gateway   = 192.168.1.1

  Is this correct (y|n): y



Alert Setup
  Email Enabled      = y
  Email Address      = me@my.com
  SMTP Server        = 192.168.1.5

  Is this correct (y|n): y


Serial Port Setup
  Serial Port Baud      = 9600
  Serial Port Data Bits = 8
  Serial Port Parity    = none
  Serial Port Stop Bits = 1


  Is this correct (y|n): y



User Setup
  User Name        = admin
  User Permissions = cuar

  Is this correct (y|n): y



This script will now update RSC, continue? (y|n): y
Updating flash, this takes a few minutes
........................................
........................................
........................................
........................................
........................................
........................................
........................................
...........................
Download completed successfully

Resetting RSC (takes about 90 seconds): DONE
Setting up server to update RSC date on boot: DONE
Setting up server hostname: DONE
Setting up ethernet interface: DONE
Setting up e-mail alerts: DONE
Disabling pager alerts: DONE
Disabling modem interface: DONE
Setting up serial port interface: DONE
Adding user to RSC:

A valid password is between 6 and 8 characters, has at least
two alphabetic characters, and at least one numeric or special
character.  The password must differ from the user's login name
and any reverse or circular shift of that login name.
Setting User Password Now ...

Password: 
Re-enter Password: 
User has been added to RSC
Resetting RSC (takes about 90 seconds):
Are you sure you want to reboot RSC (y/n)?  y

DONE
Setting up RSC date: DONE

*******************************
RSC has been successfully setup
*******************************

• If you haven’t already, make the next reboot a reconfigure reboot by running touch /reconfigure
• Now that the RSC is setup, you need to tell OpenBoot to send output to it.

eeprom input-device=rsc-console eeprom output-device=rsc-console

That’s it!!! You should now be able to telnet to the RSC (these things are old enough that there is no SSH support).

It used to be that in order to add persistent static routes in Solaris, you had to whip up your own init script that manually ran ‘route add’. Starting back in Solaris 10 11/06, Sun finally gave us a better way to do it. From the route(1M) man page:

     -p

         Make changes to  the  network  route  tables  persistent
         across  system restarts. The operation is applied to the
         network routing tables first and, if successful, is then
         applied  to  the  list  of  saved  routes used at system
         startup. In determining whether an  operation  was  suc-
         cessful, a failure to add a route that already exists or
         to delete a route that is not in the  routing  table  is
         ignored. Particular care should be taken when using host
         or network names in persistent routes, as  network-based
         name  resolution  services are not available at the time
         routes are added at startup.

So, all you need to do to add a static route for 192.168.0.0/24 to be accessed via gateway 192.168.1.1 on each boot is this:

/usr/sbin/route -p add 192.168.0.0 192.168.1.1

Currently, these routes are stored in /etc/inet/static_routes, but Sun doesn’t guarantee it will stay there. I’m not sure I like being forced to use a command when I could just edit a config file, but it’s a definite improvement!

If you make the mistake of installing Windows after Linux, it will rewrite your MBR, killing GRUB. Some might argue simply installing Windows on your computer is a mistake, but let's fix the MBR and worry about that later. ;-) In my case, my boot partition is on the first partition on the first hard drive. I use Ubuntu in my examples, but any LiveCD should do. First, boot the LiveCD - no need to boot into a GUI. Open a terminal, and become root. In Ubuntu,

sudo -s

works nicely. If you can't remember which partition is your boot partition, try

fdisk -l /dev/sda

OR

fdisk -l /dev/hda

might help jog your memory. Next, we just need to reinstall the bootloader code to the MBR. In my examples, hd0,0 is the first partition on the first disk.

root@ubuntu:~# grub
Probing devices to guess BIOS drives. This may take a long time.
       [ Minimal BASH-like line editing is supported.   For
         the   first   word,  TAB  lists  possible  command
         completions.  Anywhere else TAB lists the possible
         completions of a device/filename. ]

grub> root (hd0,0)

grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"...  16 sectors are embedded.
succeeded
 Running "install /grub/stage1 (hd0) (hd0)1+16 p (hd0,0)/grub/stage2 /grub/menu
.lst"... succeeded
Done.

grub> quit

That's it - reboot, and you'll be greeted by GRUB again!

I’m not for sure when they did it, but the RHEL folks made it a bunch easier to setup simple source policy routing. By using source policy routing, we fix the issue of firewalls freaking out when the reply packet to a host leaves a multihomed host on a different interface than what the request came in on. In prior versions, you had to setup some custom scripts, but that’s no longer the case - all the hooks are there in the OS now. In this example, imagine a CentOS host with two nics. 192.168.0.2/24 is on eth0, and 10.0.0.2/24 is on eth1. The default gateway is set to 192.168.0.1. Any host accessing 10.0.0.2 from any subnet that isn’t on 10.0.0.0/24 will have it’s reply packets sent out via 192.168.0.1. Some firewalls drop this type of traffic cough Cisco ASA’s cough. Thanks to the iproute2 package in Linux, this is easy enough to fix. RedHat has made it even easier now - we can do this in 3 steps (all performed as root):

Step 1: Create a table

We need to create a table for iproute2. Name it anything you want, and add it to /etc/iproute2/rt_tables, like so:

echo -e
"200\tCorpNet" >> /etc/iproute2/rt_tables

Step 2: Create a route

We need to create a route for eth1 that says to use our CorpNet table defined in Step 1.

echo "default table CorpNet via 10.0.0.1" >
/etc/sysconfig/network-scripts/route-eth1

Step 3: Create a rule

We need to create a rule for eth1 that says to use our route above for traffic received on eth1.

echo "from 10.0.0.2 table CorpNet" >
/etc/sysconfig/network-scripts/rule-eth1

Step 4: Restart networking

/etc/init.d/network restart

That’s it. Fire up a packet sniffer and verify your config!

So, according to NYT, the IBM buying Sun deal is near final. No sir, I don’t like it one bit. I view the merger like this: IBM is a bunch of bureaucrats that know how to sell, and Sun is a bunch of brilliant geeks that would much rather focus on cool tech than making money. While IBM had the whole “pushing Linux” thing going on, all I can think of is some commercials on TV, and Eclipse. Sun has open sourced Java, Glassfish, VirtualBox, OpenOffice, and MySQL. They basically started the multi-core movement with Niagra, and OS’s that won’t implement ZFS are struggling to copy it. If IBM buys Sun, and keeps everything as it is now – great! Sun would actually make money, and we’d still have all that cool stuff. Unfortunately, there’s a lot of clashing products, and engineers don’t generally perform well in a company like IBM. Personally, I think it stinks, and pray that something happens to stop the inevitable.

The last couple of months for me have been spent cramming 1200 pages of meaty Solaris information into my brain. At the end of my sacrifice, I have an SCSA certification to show for it. Was it worth it? For me, yes. For my employer, yes. For everyone, no. I’ve never been much for certifications. Heck, I really didn’t even have much desire to get my 4 year bachelor’s degree - I just knew that I would be better off if I had it. Fresh after graduation, I got my A+ Certification. It made me feel good, but never really did anything for me. During my job as a computer repair tech, I had the misfortune of dealing with way too many paper MCSE’s that couldn’t write a batch file. The whole experience really soured me to certifications in general – to the point of me discrediting many people that had MCP or MCSE in their email signatures. In my next job, as the sysadmin of a dialup/cable modem ISP, I had a somewhat improved experience in being exposed to CCIE’s - most of them could at least do IP subnetting without a cheatsheet. I actually learned enough RedHat Linux that I could have probably taken a month of studying and turned it into an RHCE certification, but decided against it. So, what made me turn to the dark side and get my SCSA? The idea hit me when Sun offered a promotion for half-off their training and testing package. I ran it by my boss, and he was cool with the company paying for it, and me using a little company time to do my studies. I figured what the heck, and went for it. Here’s what I learned:

• In my case, my certification was motivation. While I had worked with Solaris for the past 3 years, I really hadn’t learned that much about it. Up until a few months ago, I hated SMF (what was so bad about init scripts?), and had no idea how to use ZFS. I didn’t know where Solaris excelled, and instead used Linux for everything that I could. Now I know much more about the differences between Solaris and Linux. Details on that in a future post, but I can now make a well informed decision on what OS fits the job best, and I feel comfortable administering either.
• It’s up to you to get what you need out of the certification. If you want a piece of paper, you can go download “dumps” that contain every possible question you’ll see on the cert exam, and memorize them all. You’ll flounder and die out in the field, but that’s your prerogative. I read the materials, took notes, and at the end of each chapter I performed the actual tasks on a Solaris box at home. That type of learning sticks with you, and will likely save your arse someday.
• The test itself gets absurdly detailed at times. They expect you to memorize command line options, and give you know man pages to lookup. Sometimes the only difference between two answers is the -S in answer A versus the -s in answer B.
• The test makes you memorize things you don’t have to memorize in the field. The key to effective systems administration is good problem solving skills coupled with a good knowledge of the tools available to you. Thanks to man pages and Google, you don’t need to memorize command line options, just know the command(s) that will get you out of trouble. When all is said and done, it wasn’t the most pleasant experience ever, but it was one I’d do again. I now have confidence when faced with a Solaris problem. I must say that while I’ll stick with Linux on my desktop and laptop, I’ll actually lean towards Solaris on the server side of things. I learned a lot of things that I never would have learned had I not had to take the test, and my employer will benefit as well. By the way, if you’re looking for good material on the SCSA, go for Bill Calkin’s book and avoid Paul Sanghera’s book. I read both, and feel that Bill’s book much better prepared me for the exams. The cert doesn’t make the sysadmin, but the sysadmin can make the cert work!

Sorry for the hibernation period. I’ve just completed my SCSA exams. It wasn’t easy, but I passed! I have a data center move happening at work as well, but with my SCSA stuff (3 hours of studying each night for the past 3 weeks!) out of the way, I should have at least some time to post some content in the near future.