Flytech box Linux router training manual
June 24th 2002 Ver 1.3
1. Introduction
The Flytech boxes [1] that we
use for our routers are small PCs marketed as Point Of Sale terminals (POS).
Because they are fully functional PCs and small in size they are very useful for
many other tasks, not to mention that they cost much less than Cisco
routers.
Because we run the Linux OS on them we have much more
flexibility on how they are set up and configured. The trade off is that you do
have to plan out carefully exactly what you want them to do. In addition to
routing you can run other services on them such as a web server to display the
status of the network etc...
The Particular model of Flytech that we are
using have to PCI slots or one PCI and one ISA. Since our links out of Thimphu
use the existing microwave network to get close to our sites we have an E1 card
in one slot to connect to free E1 slices of this network. The other PCI slot has
a four port 10/100 card in it. The motherboard itself has a built in 10/100
port. This configuration gives you five 10/100 ports and one E1. You can also
get multiport E1 cards from SBE if you want to connect to several at one site.
You could also put two four port 10/100 cards in these boxes for a total of nine
ports.
You can buy them with various speed CPUs and amounts of RAM. Some
example costs are about $500 for a 500MHz 64MB, 30Gig HD, and $815 for a 850MHz
512MB 40Gig HD. The four port cards are about $340 and the single port E1 cards
are about $900.
2. Installation
The hardware part of the installation is fairly straight
forward. Just find a shelf to put the box on and connect a monitor and keyboard.
You should power it with a UPS or inverter since the network must stay up even
when there is a power failure. The power supply auto detects 120 or 240 Volt
AC.
Next connect the network cables to free ports on the back. Since the
microwave network uses 75 ohm coax to provide the E1 connections and the E1 card
uses UTP cable we need a little Balun to convert between them. Mainly at this
stage you just need to keep track of which ports you plug the cables into.
Ideally the port on the motherboard would be eth0 and the four on the quad port
card numbered from left to right become eth1, eht2, eth3, and eth4. The E1 port
is hdlc0.
3. Configuration
3.1 Introduction to networking
Well I was looking for a quick introduction to networking and routing,
and found a couple, [2], and [3] , but
they weren't quite what I was looking for. So for now this will have to do. OK
there are other protocols than IP but our network is an IP network like the
Internet so that is what we will cover here.
First of all every device
that you can talk to has an IP address, which is like a telephone number. Thats
how you call the device. An IP address is four bytes long, that means it has 32
bits total. IP addresses are written with the byte values separated by dots. For
example: 202.144.156.240, or 10.10.5.128
Next there is the netmask it
tells you who is on the same network with you. It's like your local calling
area. You can call anyone on your local network directly but if you want to talk
to someone on another LAN somewhere else then you will have to go through a
gateway. This is like an operator assisted call. After you pass your packet to
the gateway it figures out how to deliver it. All a netmask does is tell
you what portion of the IP address contains the host address on the local
network. The rest of it is just an address for local network itself, and as long
as that part of the address is the same as the local network address then no
gateway is needed to deliver it because all machines on the LAN are listening
for packets sent to them. Here are some example netmasks: 255.255.255.0,
255.255.0.0, 255.255.255.240, 255.255.255.255
If you display these
netmasks as binary numbers then you will see that the left hand side is all ones
and some number of bits on the right are all zeros. The first is a common
class C subnet which means that the last byte contains the host portion of the
address. So you can have hosts numbered from 1 to 254 (addresses 0 and all ones
are special and can't be used for hosts). The next one is a class B which has a
16 bit host portion so it can have about 64,000 hosts on the same network. Next
we have a small subnet that ends in .240 this gives you four bits of host
address space so you could have hosts numbered from 1 to 14, remember to skip
the first and last values. Here is a little trick. If you take 2 to the power of
the number of bits you want and subtract that from 256 it will give you the
netmask you need. Lets try it:
If we want a 5 bit subnet then 2^5 is 32.
So 256 - 32 is 224, so our full netmask would be 255.255.255.224. Now don't
forget we can't use all 32 addresses in our 5 bit subnet. The usable space is
always 2^N-2 so for this case it would be 2^5-2 which is 23-2 or 30. So if you
wanted to have say, 31 machines on a subnet you would have to go up to a 6 bit
netmask which would give you 2^6-2 = 62 possible addresses.
The two
mysterious addresses that you can't use are the ones where the host portion are
all zeros and all ones. The first is the network address and the other is the
broadcast address. The broadcast address is used when you need to ask all
devices something. So actually any device will respond to packets sent
specifically to their address and also to packets sent to the broadcast
address.
Last but not least you sometimes need to specify a gateway to
handle traffic outside of your LAN. The gateway address must always be for some
machine in your LAN so it has a local address. The way gateways work is that
they another interface on another LAN or even several interfaces and they route
packets between them. Its like a telephone exchange routing the long distance
calls onto the microwave link.
So when you set up a network interface you
need to specify several things, an address, netmask, and probably a broadcast
address though some configuration software will calculate the broadcast address
for you. Then you may also want to set a gateway for the LAN this interface is
attached to.
3.2 Basic commands
There are some simple commands to set and change
these values for an interface. First each interface has a name such as eth0,
eth1, hdlc0, etc... You can list all the active interfaces by using the command
ifconfig. Here is an example:
# ifconfig
eth0 Link encap:Ethernet HWaddr 00:07:50:CA:C3:C6
inet addr:202.144.156.230 Bcast:202.144.156.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3830 errors:13 dropped:0 overruns:0 frame:13
TX packets:1779 errors:1 dropped:0 overruns:0 carrier:187
collisions:572 txqueuelen:100
RX bytes:1550803 (1.4 MiB) TX bytes:216743 (211.6 KiB)
Interrupt:3 Base address:0x100
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2822 errors:0 dropped:0 overruns:0 frame:0
TX packets:2822 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:281570 (274.9 KiB) TX bytes:281570 (274.9 KiB)
#
This
shows two interfaces, eth0 and lo. Eth0 is the one real connection to the
outside world and lo is known as the loopback interface it is a way for
applications and other processes on the computer to talk to themselves in the
same way applications on two different computers would. The loopback interface
is convenient and necessary so you will always see it listed. Under the listing
for eth0 you can see that it's address is 202.144.156.230, and its netmask is a
class C. There are also some simple statistics tracking the numbers of errors
and total packets sent.
So lets say you wanted to set this interface up
for another network you could type a command like:
ifconfig eth0 10.10.10.130 netmask 255.255.255.0 broadcast 10.10.10.255
This would set the new address, netmask and broadcast for eth0 to
10.10.10.130, 255.255.255.0, and 10.10.10.255 respectively.
You can also
check to see if an interface is physically there and has the correct driver by
typing something like:
ifconfig eth0 up
If you don't get any errors then it's ready to
be assigned an address etc, like the example above. You could also now type
ifconfig to list all the interfaces again. If you want to temporarily turn
off an interface then you can type something like this:
ifconfig eth0 down
But don't forget if you're talking
to the machine over the network then you can't turn off the interface that your
talking to it on. Also, if you re re-boot the machine the interface will most
likely come back up if that is how the boot scripts are configured.
For the
complete documentation on ifconfig you can type "man ifconfig" at a
prompt.
Another handy command is route it lets you add among other things
a gateway. Here is an example:
route add default gw 202.144.156.254
This sets the default
gateway to 202.144.156.254. Route can add many other kinds of routes and a full
discussion of routing is beyond the scope of this document. There are some other
introductions to networking listed in the appendix. For a complete description
of route type "man route" at a prompt.
another handy command is netstat.
"netstat -rn" will list your routing table. Example:
# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
202.144.156.0 0.0.0.0 255.255.255.0 U 40 0 0 eth0
0.0.0.0 202.144.156.254 0.0.0.0 UG 40 0 0 eth0
#
This
is a very simple table with two entries. The first says that the class C network
can be reached through eth0, and the second says that any other destination you
should go though the gateway 202.144.156.254 which is also reachable through
eth0.
3.3 Config files
Now when a computer boots up one of the last things it
does is read some config files that tell it who it is and what it's address is
etc... If you make some changes with ifconfig or route and reboot the computer
they will all be forgotten when it reads these files and sets itself up with the
old information. So to make any permanent changes you need to edit the actual
config file that controls these settings. In Debian the file that sets up the
network interfaces is /etc/network/interfaces. Here is an example file:
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
# The loopback interface
# automatically added when upgrading
auto lo
iface lo inet loopback
# The first network card - this entry was created during the Debian installation
# (network, broadcast and gateway are optional)
# automatically added when upgrading
iface eth0 inet static
address 12.33.21.201
netmask 255.255.255.0
network 12.33.21.0
broadcast 12.33.21.255
gateway 12.33.21.254
iface eth1 inet static
address 198.0.0.254
netmask 255.255.255.0
network 198.0.0.0
The
paragraph for eth0 sets the address, netmask, network, broadcast, and gateway.
The paragraph for eth1 lets the system calculate the network, and broadcast
addresses. After you have made your changes you could type a command like "ifup
eth0" to bring up a interface. To take down an interface the command is ifdown
eth0, or eth1 etc... Sometimes you want to edit /etc/network/interfaces and then
update the values for a particular interface. You need to be careful because
most of the time you will be talking on the interface you want to change. So a
command like this works pretty well:
ifdown hdlc0; ifup hdlc0
This is how you write
several commands on one line, they are separated by semicolons ";". The
commands are executed one after the other when you press enter. So you
connection to this machine goes down and then comes back up with the new
information in the interfaces file. Often after you make a change like this you
also need to update the interface on your local machine before it can talk to
it's partner again. Of course a simple way to update everything on the machine
is to just reboot it.
4. Maintenance
There is really not much maintenance that needs to be
done on a simple router like the Flytech boxes unless you want to make changes
to the network. All the routes are static right now because it is a small pilot
project but you could run routd or BGP or almost any routing protocol if you
felt it was necessary.
4.1 Adding new interfaces
Since these routers have five 10/100
interfaces on them you could plug in additional networks and route between them.
Say for example that you wanted to install another radio in Dochula on a
different subnet like 10.4.0.0 going off in another direction. Then you would
find an unused interface on the Flytech box there and add config information for
it in the interfaces file. Lets say that we'll use eth3. It might look something
like this:
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
# The loopback interface
# automatically added when upgrading
auto lo
iface lo inet loopback
# The first network card - this entry was created during the Debian installation
# (network, broadcast and gateway are optional)
# automatically added when upgrading
iface eth0 inet static
address 12.33.21.201
netmask 255.255.255.0
network 12.33.21.0
broadcast 12.33.21.255
gateway 12.33.21.254
iface eth3 inet static
address 10.4.0.0.1
netmask 255.255.255.0
network 10.4.0.0
broadcast 10.4.0.255
Sometimes
you add a subnet out in the field which is served by a radio repeater. The
clients for that repeater need a gateway in their network so that they can talk
to the rest of the system or the world. The Radios themselves don't need to use
routes to figure out which way to send packets. They do something called
bridging which is done only using MAC addresses (the documentation for the
radios refer to these as "node" addresses). Any bridging is invisible to the IP
address world. You can have any number of IP subnets served by a group of
radios and the packets will be delivered correctly. Because of this it turns out
that it is convenient to assign the radios and Flytech boxes in an area with
there own subnet separate from all the client subnets. But then what about the
gateway for a group of clients? You can't make it their parent repeater because
it already has an IP address in a different subnet, and the same goes for the
router interface on the Flytech box for the same reason. What to do? ;-) ...
Well you could connect another interface to a hub that had the original two
cables on it and configure that interface to be the chosen gateway address for
the client's subnet. But you would still need a new interface for each new
subnet and since there are only five on the Flytech boxes you would quickly run
out. luckily there is something called "Virtual Interfaces" to solve this
problem. It's a way make one interface act like many, all with different address
and even different subnets. The way you specify a virtual interface is to first
find the real interface that you want to add a new address and/or subnet to.
Lets say that we want to add a gateway for the subnet 10.5.1.0 to eth2. Then the
new virtual interface could be called eth2:0 or eth2:1, or eth2:5. or eth2:99,
it's up to you. Here is how the paragraph of the interfaces file might look:
iface eth2:5 inet static
address 10.5.1.254
netmask 255.255.255.0
network 10.5.1.0
broadcast 10.5.1.255
So
we now have a new virtual interface on eth2 that acts as a gateway for the
example client's subnet. Because the radios do bridging they will soon
learn of the new interface automatically and deliver packets to the new gateway
as needed. Then the router will look in it's routing table to direct them to the
proper destination.
By the way, each time you add an interface weather it
is real or virtual, a route is automatically added to the routing table to
direct packets for the new subnet to that interface.
4.2 Adding routes
Sometimes you need to let other machines know how to
get to particular subnets. For example, in Thimphu there are two Flytech boxes.
One talks to the Limukha area and the other talks to Gelephu. In Limukha all the
client subnets are in the range 10.2.y.x, and in Gelephu they are 10.3.y.x where
"y" is some small number from 0 to 3 or so. But since there are no interfaces on
these Flytech boxes that are set up for these subnets there are no routes to
tell them where to send packets to for all these clients. When this happens a
router will use the default route (If it has one) to some other gateway. This is
like saying "I have no clue where this packet goes, please deliver it for me."
But in this case some of the client packets need to go to the box on the other
end of the E1 line and the other clients are reached via the other Flytech box
in Thimphu.
So we need to add some additional routes that could look like
this:
route add -net 10.2.0.0 netmask 255.255.255.0 gw 10.1.0.2
route add -net 10.2.1.0 netmask 255.255.255.0 gw 10.1.0.2
route add -net 10.2.2.0 netmask 255.255.255.0 gw 10.1.0.2
These
direct packets for the three class C subnets 10.2.0.0, 10.2.1.0, and 10.2.3.0 to
the box whose IP address is 10.1.0.2 which is the router on the other end of the
E1 in this case. But we still need to deliver packets for the other set of
clients in Gelephu, which are handled on the other box. So we add some more
routes like this:
route add -net 10.3.0.0 netmask 255.255.255.0 gw 10.0.0.254
route add -net 10.3.1.0 netmask 255.255.255.0 gw 10.0.0.254
route add -net 10.3.2.0 netmask 255.255.255.0 gw 10.0.0.254
And
these routes send these subnets to the box whose IP address in 10.0.0.254. Now
you may wonder how this box knows which interface to use to get to these
gateways. Well remember any gateway you list must already be on a subnet that is
attached to a real or virtual interface on this box, and whenever you configure
an interface a route is automatically added to the table to direct packets
destined for that subnet to the correct interface. If you make a mistake and the
gateway is not directly reachable then you will get a "Network is unreachable"
error message.
Here is a little trick to making your routing tables easer
to read and maintain. notice that the Flytech boxes in Thimphu don't really need
to know exactly what class Cs are being used in Limukha and Gelephu, they just
need to know that all packets in the 10.2.y.x range go to Limukha and all the
ones in the 10.3.y.x range go to Gelephu. We don't even have to worry about the
fact that there are network addresses and broadcast addresses at the start and
end of each range, all we need to do is ship all that stuff upstream to the
router that really handles it. So we can replace those six routes with just two
like this:
route add -net 10.2.0.0 netmask 255.255.0.0 gw 10.1.0.2
route add -net 10.3.0.0 netmask 255.255.0.0 gw 10.0.0.254
Notice
that we changed the netmask to be a whole class B this includes all packets in a
y.x range. Then each time you add a new client subnet on the boxes for Limukha
or Gelephu you don't need to do a corresponding update in Thimphu because it is
already handled. Also the fewer routes in your routing table the faster a box
can move packets. Though these tables are so small anyway it doesn't matter
much. :-)
5. Operation
There is not much that needs to be done to the routes for
day to day operation. However there are some precautions you need to take to
keep them running smoothly. First as you know most computers are not set up to
handle random power failures. All routers should be on UPSs or inverters if they
are powered off of the telco battery bank in a switching center. This means that
you can't just turn them off by throwing the MCB with out asking them to
shutdown first. If you do this it is like playing Russian roulette with your
file system. Most of the time the errors caused by a loss of power can be
automatically corrected... but not always. Sometimes the machine will not
comeback up and require manual intervention. Usually this means typing in the
root password and running fsck on the corrupted file system and typing "Y" for
each major repair that it needs to do. This also usually means that some date is
lost. If you are lucky it was just temporary data in a scratch file somewhere.
Once in a while though you are not so lucky and a critical file is damaged. If
this happens the system will no longer run even after the file system is
repaired. Now you have to reload the who system from scratch. This will probably
take a whole day or more depending on how far you have to drive. No phone calls
can be made during this time and lots of people will be upset. So you need to
educate the maintenance people not to go throwing MCBs on the Flytechs even if
they think they have a good reason to. By the way, this is exactly the same
situation you have with the servers at Druknet.
There are enhancements
that could be made to the Flytechs to partially or completely remedy this
situation. For example they could be converted to use journaling file systems.
this means they would roll forward or backward any partially written updates
more like a data base does. Also you could install a smaller Linux system that
could all fit in RAM, the disk would just be used at boot time to initialize the
RAM disk and then all updates would be done there. Of course if you make any
configuration changes you also need to copy them back to the disk or they will
be lost on the next boot. If you are worried that the hard drives might
eventually fail you can put the system on a 128MB or 256MB Compact Flash (CF)
card and boot off of that since the Flytechs do support this option.
5.1 Monitoring the E1 interface
There is a handy utility called sbectl
that lets you view the state of the hdlc interface or display stats on it. type
sbectl -h for help.
6. Backup
The basic idea here is to save a known working system to a CD
or a hard disk on another computer and later if you need to, reformat your drive
and write it back to disk. To do this you need a Debian rescue disk, but not
just any Debian rescue disk. You need one specially configured for the Flytech
box. This is because the Flytech boxes don't have a normal floppy drive. Instead
they have a LS-120 drive which can also read floppy disks, but Linux usually
expects floppies to be read from device /dev/fd0 and the LS120 is an IDE device
so it has a name like /dev/hda or /dev/hdb. This causes lots of problems unless
you make or have disks that are set up correctly, then you don't even notice the
difference. What you should do then is make master copies of these rescue disks
and two or three sets for emergency use. Then you can always make new ones from
the master which is never used for anything else.
Backing up is pretty
easy first we need to do a df to see how big the system is and where we might
find some extra disk space.
#df
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/hdc4 29459052 485388 27477212 2% /
/dev/hdc3 9614148 5949876 3175896 66% /aux
/dev/hdc1 7746 3375 3971 46% /boot
So
what we want to back up is / (root) and /boot. /Aux is just extra storage and is
not part of the OS though you might want to back it up separately if you
value it's contents. We see that the OS is about 500MB which fits nicely on a
CD. Actually when compressed it may only take up about 300MB so you could fit
two on one CD. To get the cleanest backups of a system it should not be in use,
there are always a few files being updated like logs and temporary scratch files
etc. These are fairly unimportant and you really rather not have to take the
routers down to back them up anyway so I would suggest you do live backups. So
it goes something like this. Lets say that /aux has enough free space so we can
put the image there.
cd /aux
tar czlf flyback1.tgz / /boot
So we cd to /aux
and use the tar command to make a compressed archive of all the important
directories there.
Then you can ftp this file off to some other system to
store or burn onto a CD. Now you may not want to copy 300MB of data each time
you change an interface on the boxes. So since nearly all the config files are
in /etc you could just copy that directory and back it up like this:
cd /aux
tar czf flyetc1.tgz /etc
Then after you load the
latest full backup you load the latest /etc backup and your done.
In June
2002 I automated the backup system a bit. Now every week full system images are
made along with MD5 checksums on each machine. Then these are copied to most of
the other machines. The exception is the two remote machines don't get each
others copy because I didn't want to take up too much bandwidth on the E1 links.
All the backups on each machine are in /aux/backups. Older copies are also kept
up to about a month. Now if you need to do a restore you can copy the
appropriate image from almost any working Flytech box to your laptop. You can
find the backup scripts in the appendix.
To restore It's a little more
complicated because your starting from scratch. Lets assume that your disk is
not physically damaged but is corrupted beyond repair. You may not need to
repartition it but if you do then it's best to keep the partition numbers
unchanged, ie. If root was partition one before it needs to stay partition one.
See the appendix for the originals [8.3].
But basically make a small 5MB partition at the start, then slightly less than
half of the disk, then a 256MB swap partition, and last, whatever is left. But
it's quite likely that you can skip this step because partition tables rarely
get corrupted.
Before we get started you will need a laptop with a CD
drive and a network interface running a web server. This is where the backup
images will come from. They will either be on a CD or the hard drive and should
be findable by the web server.
OK to start, take the special LS120 rescue
disk and boot off of it. You might need to check that the BIOS boot sequence is
LS120,C. I think it's possible to also create a set of 1.44 MB floppies but it's
less convenient. When you get to the "Debian GNU/Linux 2.2!" screen and
the boot prompt at the bottom, you can just hit enter. Then after a little while
you get to another informational screen: Then follow these
steps:
1) Press enter to get to the main menu.
2) Press Enter twice to
configure your keyboard for the US standard.
3) If you don't need to
partition your drive again then it's safe to activate your swap partition so you
would press enter
about four times to do this and return
to the menu.
4) You will probably want to initialize one or all of the Linux
partitions. Select this item if it's not at the top and it will let you
chose
which partitions to initialize. You must select the
one that will be the root partition first. Answer no when it asks if you want to
retain kernel 2.0
compatibility, and press enter for the rest. It will mount the first one as
root. Then go on to the rest, the very small one should be
/boot and the other large one should be /aux. See the appendix for the current
partition layouts of the Flytechs.
5) Now it should ask you to install kernel
and modules. So select that item by pressing enter. Select IDE floppy for the
installation
medium, and then the
line that says LS-120. Put the disk in the drive and press enter until it starts
installing and returns you to the menu.
Note that if your floppy is write protected it will fail to mount, so you will
have to make it writable and try again. This is a bug in the
Debian script.
6) Configure the device
driver modules. Since all the necessary modules are already compiled into the
kernel you can just press enter here until
you're back at the main menu.
7) Configure the network. It will ask you for a
host name which will be over written with the backup image later so you can just
press enter for the default. Since you
probably will not have a DHCP server handy you should answer no at the next
question and do a
manual configuration instead. Next type in a temporary IP address. Then press
enter for the default Class C netmask. Thats all you
really need, no gateway or nameservers are necessary so
answer those questions as you like.
8) Install the base system. Here is where
we get the full backup. Select network as the installation medium, this is the
last item on the menu. It will tell you what file(s) it wants to download so
press enter. Then type Control-U to erase the first line on the next page so you
can type in the IP address of the laptop a "/" then the rest of the URL to the
file which must be renamed "base2_2.tgz". You could also make a symlink of that
name to the actual filename. For example if base2_2.tgz is in the DocumentRoot
of the web server then you could use a URL like this: http://202.144.156.39/
Then hit tab until "OK" is highlighted and press enter. It should start
downloading the compressed tar archive.
9) Remove the floppy if you haven't
done so already
10) Select Make Linux Bootable Directly from hard disk. You
now have two choices, Install Lilo in the MBR (which should work) or install
Lilo in the root partitions boot sector. Hit enter for the first choice. If you
get no complaints then you can reboot
11) Reboot the machine, You will notice
that it continues the tail end of the Debian configuration. This is a remnant of
the rescue disks install
process. Don't
bother to answer any questions, just press ALT-F2 to switch to the next virtual
console and log in as root.
12) cd to / and rm the file @longlink
13) cd
to /etc and restore the correct inittab file like this:
cd /etc; mv inittab.real inittab
14) Reboot again, and
your done!
Now you may want to copy some of the more recent config file
images onto the system.
7. Other services
On the Gelephu box there is also an Apache web server
used just to display the radio statistics. It's config files are in /etc/apache.
These stats are collected by Cricket which is similar to MRTG. Cricket's config
files are in /home/cricket/cricket-config and the online documentation is in
/home/cricket/cricket/doc. To view the radio pages
goto:
http://10.0.0.254/~cricket/
8. Appendix
8.1 Backup scripts
The stage zero scripts are all about the same just
the file name is different. They rotate the images to older names, make the tar
archive and the MD5 checksum. They are run by cron every Sunday at 4:00
AM. Here is the one from bt1: #/bin/sh
# Stage zero of the Flytech backup image copying script
# Clif Cox June 2002
cd /aux/backups
mv -f bt1arc.1.tgz bt1arc.2.tgz; mv -f bt1arc.1.sum bt1arc.2.sum
mv -f bt1arc.0.tgz bt1arc.1.tgz; mv -f bt1arc.0.sum bt1arc.1.sum
mv -f bt1arc.tgz bt1arc.0.tgz; mv -f bt1arc.sum bt1arc.0.sum
mv -f bt2arc.1.tgz bt2arc.2.tgz; mv -f bt2arc.1.sum bt2arc.2.sum
mv -f bt2arc.0.tgz bt2arc.1.tgz; mv -f bt2arc.0.sum bt2arc.1.sum
mv -f bt2arc.tgz bt2arc.0.tgz; mv -f bt2arc.sum bt2arc.0.sum
mv -f bt3arc.1.tgz bt3arc.2.tgz; mv -f bt3arc.1.sum bt3arc.2.sum
mv -f bt3arc.0.tgz bt3arc.1.tgz; mv -f bt3arc.0.sum bt3arc.1.sum
mv -f bt3arc.tgz bt3arc.0.tgz; mv -f bt3arc.sum bt3arc.0.sum
mv -f bt4arc.1.tgz bt4arc.2.tgz; mv -f bt4arc.1.sum bt4arc.2.sum
mv -f bt4arc.0.tgz bt4arc.1.tgz; mv -f bt4arc.0.sum bt4arc.1.sum
mv -f bt4arc.tgz bt4arc.0.tgz; mv -f bt4arc.sum bt4arc.0.sum
tar czlf bt1arc.tgz / /boot
md5sum bt1arc.tgz > bt1arc.sum
And here is the one from bt2:
#/bin/sh
# Stage zero of the Flytech backup image copying script
# Clif Cox June 2002
cd /aux/backups
mv -f bt1arc.1.tgz bt1arc.2.tgz; mv -f bt1arc.1.sum bt1arc.2.sum
mv -f bt1arc.0.tgz bt1arc.1.tgz; mv -f bt1arc.0.sum bt1arc.1.sum
mv -f bt1arc.tgz bt1arc.0.tgz; mv -f bt1arc.sum bt1arc.0.sum
mv -f bt2arc.1.tgz bt2arc.2.tgz; mv -f bt2arc.1.sum bt2arc.2.sum
mv -f bt2arc.0.tgz bt2arc.1.tgz; mv -f bt2arc.0.sum bt2arc.1.sum
mv -f bt2arc.tgz bt2arc.0.tgz; mv -f bt2arc.sum bt2arc.0.sum
mv -f bt3arc.1.tgz bt3arc.2.tgz; mv -f bt3arc.1.sum bt3arc.2.sum
mv -f bt3arc.0.tgz bt3arc.1.tgz; mv -f bt3arc.0.sum bt3arc.1.sum
mv -f bt3arc.tgz bt3arc.0.tgz; mv -f bt3arc.sum bt3arc.0.sum
mv -f bt4arc.1.tgz bt4arc.2.tgz; mv -f bt4arc.1.sum bt4arc.2.sum
mv -f bt4arc.0.tgz bt4arc.1.tgz; mv -f bt4arc.0.sum bt4arc.1.sum
mv -f bt4arc.tgz bt4arc.0.tgz; mv -f bt4arc.sum bt4arc.0.sum
tar czlf bt2arc.tgz / /boot
md5sum bt2arc.tgz > bt2arc.sum
The stage one scripts are run about 15 minutes
later. They use the ssh copy program "scp" to transfer the files from
neighboring machines. We run the stage one and two scripts as user flyback
instead of root because scp needs to login to another machine without using a
password. It does this by using a cryptographically strong key that is shared by
all machines. This is probably secure enough for a non-privileged user. But if
we had used root instead then anyone who accidently found a root shell on one
machine could get root on any other without typing the password, probably not
what we want... Here is the stage one script from bt3:
#/bin/sh
# Stage one of the Flytech backup image copying script
# Clif Cox June 2002
cd /aux/backups
scp -pBq "bt4:/aux/backups/bt4arc.*" .
scp -pBq "bt1:/aux/backups/bt1arc.*" .
The
stage two script is run about 45 minutes after stage one. It copies a image from
its neighbor that was transfered during stage one. Stage two script from
bt3:
#/bin/sh
# Stage two of the Flytech backup image copying script
# Clif Cox June 2002
cd /aux/backups
scp -pBq "bt1:/aux/backups/bt2arc.*" .
8.2 Making a rescue floppy
As was mentioned in section 6 the Standard
issue Debian rescue floppies have to be modified a bit to work with the Flytech
boxes. I describe how to make new ones from scratch here, and yes it is a
hassle. You relay don't want to do this if you can avoid it. So please take good
care of your master rescue disks and only use copies of them to reload your
machines if you need to. If something happens to your master disks you can get a
copy of the image [4] Which can
be copied to an LS-120 disk like so:
1) Uncompress the
image
bunzip2 flyrescue.bin.bz2
2)
Put an LS-120 disk in the drive, we'll assume that it is device /dev/hdb
3)
Copy the image to the LS-120
dd
if=flyrescue.bin of=/dev/hdb bs=1M
4) Reboot the machine to test the new
rescue floppy
That said, this procedure involves getting some images from
the www.debian.org site and building a 2.2.19 version Linux kernel from the
source. Some of the steps involved are outside the scope of this document. Try
to enlist the help of a Linux person.
So the files we will need
from the debian site are: the 2.9MB rescue.bin [7],
or a tar file of its contents [8],
and drivers.tgz [9].
From
www.kernel.org you need the source to Linux kernel version 2.2.19. The extra
utilities that you will need are mkdosfs to format an LS-120, and syslinux to
install the boot block on the new rescue disk. Of course you will also need some
LS-120 floppy disks which are probably available by mail order or from
Bangkok.
First we need to build a new kernel, so un-tar the source in a
convenient location and configure it with at least these drivers linked
in:
initrd, ramdisk, loop, msdos, fat, elf, ext2fs, procfs, IDE disk support,
IDE floppy support, and in the Ethernet 10/100 area:
CONFIG_EEXPRESS_PRO,
CONFIG_DEC_ELCP, CONFIG_EEXPRESS_PRO100.
These ethernet drivers should
cover the possible chip sets on the Flytech mother boards but I haven't been
able to test all of them. If the ethernet port doesn't come up then you might
try loading different drivers from that menu item on the rescue disk. For the
standard 2.2.19 modules that come with the debian rescue disk to load correctly
it's important to compile your kernel with the "Set version information on all
symbols for modules" turned on. Of course there are many other kernel options
that must be set correctly but the defaults should work fine. It might be good
to start with the kernel config file I used [6]. A
little more documentation on this part is in the readme.txt file on the rescue
disk itself.
If you are using LS-120 disks:
1) Put an LS-120 disk
in the Flytechs floppy drive.
2) Determine the device name of the LS-120
drive. You could use the following command:
find /proc/ide -type f -exec egrep -l "LS.?120" {}
\;
You should get one line of output
that has an hda, hdb, hdc, etc in it. This is the device name.
3) From a root
shell format it using the mkdosfs command. Here we are assuming that the LS-120
is device hdb, don't forget the /dev/ :-)
mkdosfs -v -c -I -n Flytech_Rescue
/dev/hdb
The "-c" option checks for bad
blocks and takes a long time for a LS-120 disk but it's probably a good idea to
double check your rescue
disks integrity.
5) mount the new floppy with the
command:
mount -t msdos /dev/hdb
/floppy
6) Download or find the 2.9MB rescue floppy image on the debian site
or get a copy here [7].
Alternatively get the tar file of its contents [8] .
7)
If you have the rescue.bin image then mount it using a loop mount like
this:
mount -t msdos rescue.bin /mnt/
-o ro,loop
8) Cd into your new floppy disk
cd /floppy
9) Copy the files from the rescue image here
like this:
cp -r /mnt/* .
10) Or if
you have the tar file instead then do this:
tar xzf /some/path/to/rescue-2.9.tgz
11) Now copy your new
kernel to the floppy:
cp
/some/path/to/your/linux/arch/i386/boot/bzImage linux
12) Then we need to run
the rdev.sh command like it says in readme.txt. If you don't do this the console
will lock up with some obscure error
about an insmod unix.o failing over and over
again.
./rdev.sh
13) OK now we make
a place for the kernel image and drivers to go
mkdir images-1.44
14) And copy these files
there:
cp drivers.tgz rescue.bin
images-1.44
Yes it seems funny but you
need a copy of rescue.bin inside of rescue.bin! There is probably a more elegant
way to do this but there is
plenty of space on the LS-120.
15) Now
we just unmount the floppy and run syslinux, and we can reboot to test
it.
cd; umount
/floppy
syslinux /dev/hdb
By the
way there are some references to /dev/fd0 (the usual floppy drive) in the config
file syslinux.cfg These won't be used for the restore procedure that we outline
in section 6 but if you'd like to use these for more general purposes then you
should change all the /dev/fd0 and /dev/fd1 to /dev/hdb etc...
8.3 Flytech Partition table
info
If in the unlikely event that the partition table of a Flytech box
becomes corrupted then you will find this section helpful in recreating it. Note
that the Backups of the file systems expect the root partition to have a
particular number and so on. So it is important that when you recreate it that
the numbering of the partitions is exactly the same as listed here, though the
sizes could be a little different if you like. By the way as of June 23rd BT1
became a spare router and now has a copy of bt3's image on it.
BT1 partition table:
Disk /dev/hda: 255 heads, 63 sectors, 5005 cylinders
Units = cylinders of 16065 * 512 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 1 8001 83 Linux
/dev/hda2 2 1217 9767520 83 Linux
/dev/hda3 1218 1279 498015 82 Linux swap
/dev/hda4 1280 5005 29929095 5 Extended
/dev/hda5 1280 5005 29929063+ 83 Linux
BT2 partition table:
Disk /dev/hda: 255 heads, 63 sectors, 2491 cylinders
Units = cylinders of 16065 * 512 bytes
Device Boot Start End Blocks Id System
/dev/hda1 1 1 8001 83 Linux
/dev/hda2 2 1225 9831780 83 Linux
/dev/hda3 1226 1258 265072+ 82 Linux swap
/dev/hda4 1259 2491 9904072+ 83 Linux
BT3 partition table:
Disk /dev/hda: 255 heads, 63 sectors, 5005 cylinders
Units = cylinders of 16065 * 512 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 1 8001 83 Linux
/dev/hda2 2 1217 9767520 83 Linux
/dev/hda3 1218 1279 498015 82 Linux swap
/dev/hda4 1280 5005 29929095 5 Extended
/dev/hda5 1280 5005 29929063+ 83 Linux
BT4 partition table:
Disk /dev/hda: 255 heads, 63 sectors, 5005 cylinders
Units = cylinders of 16065 * 512 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 1 8001 83 Linux
/dev/hda2 2 1217 9767520 83 Linux
/dev/hda3 1218 1279 498015 82 Linux swap
/dev/hda4 1280 5005 29929095 5 Extended
/dev/hda5 1280 5005 29929063+ 83 Linux
9. References
[1] Flytech Group International: http://www.flytech.com/
[2]
Linux Headquarters: Network Configuration Using the Command Line: http://www.linuxheadquarters.com/howto/networking/networkconfig.shtml
[3] Introduction to Networking: http://www.thelinuxreview.com/howto/intro_to_networking/book1.htm
10.1 Supporting files for
building rescue disks
[4] The compressed rescue floppy image:
http://www.bhutan-notes.com/clif/flyrescue/flyrescue.bin.bz2
[5] The a Linux kernel with the IDE floppy driver built in: http://www.bhutan-notes.com/clif/flyrescue/linux
[6] The config file for the above kernel: http://www.bhutan-notes.com/clif/flyrescue/kernel-config.txt
[7] The Original Debian rescue image: http://www.bhutan-notes.com/clif/flyrescue/rescue.bin
[8] A tar file of the contents of the rescue image: http://www.bhutan-notes.com/clif/flyrescue/rescue-2.9.tgz
[9] Additional drivers that are probably not needed: http://www.bhutan-notes.com/clif/flyrescue/drivers.tgz