Archive for the System Category

Incremental backup scripts

Posted in System with tags , , on May 9, 2008 by wsjoung
# creates backups of essential files
DATA="/home /root /usr/local/httpd"
CONFIG="/etc /var/lib /var/named"
mount /mnt/backup
set $(date)
if test "$1" = "Sun" ; then
        # weekly a full backup of all data and config. settings:
        tar cfz "/mnt/backup/data/data_full_$6-$2-$3.tgz" $DATA
        rm -f /mnt/backup/data/data_diff*
        tar cfz "/mnt/backup/config/config_full_$6-$2-$3.tgz" $CONFIG
        rm -f /mnt/backup/config/config_diff*
        # incremental backup:
        find $DATA -depth -type f \( -ctime -1 -o -mtime -1 \) -print > $LIST
        tar cfzT "/mnt/backup/data/data_diff_$6-$2-$3.tgz" "$LIST"
        rm -f "$LIST"
        find $CONFIG -depth -type f  \( -ctime -1 -o -mtime -1 \) -print > $LIST
        tar cfzT "/mnt/backup/config/config_diff_$6-$2-$3.tgz" "$LIST"
        rm -f "$LIST"
# create sql dump of databases:
mysqldump -u root --password=mypass --opt mydb > "/mnt/backup/database/mydb_$6-$2-$3.sql"
gzip "/mnt/backup/database/mydb_$6-$2-$3.sql"
umount /mnt/backup

>> Backup script for Linux using tar and find


How to attach a new hard disk to ubuntu linux

Posted in System with tags , , , on May 8, 2008 by wsjoung

root@server-tribeca:/etc# fdisk -l

Disk /dev/sda: 40.0 GB, 40020664320 bytes
255 heads, 63 sectors/track, 4865 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x9dc96e9e

Device Boot Start End Blocks Id System
/dev/sda1 * 1 4689 37664361 83 Linux
/dev/sda2 4690 4865 1413720 5 Extended
/dev/sda5 4690 4865 1413688+ 82 Linux swap / Solaris

Disk /dev/sdb: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00a080f4

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 13 104391 83 Linux
/dev/sdb2 14 14471 116133885 83 Linux
/dev/sdb3 14472 14593 979965 82 Linux swap / Solaris

root@server-tribeca:/etc# fdisk /dev/sdb

The number of cylinders for this disk is set to 14593.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs

Command (m for help): d
Partition number (1-4): 1

Command (m for help): d
Partition number (1-4): 2

Command (m for help): d
Selected partition 3

Command (m for help): d
No partition is defined yet!

Command (m for help): n
Command action
e extended
p primary partition (1-4)
Partition number (1-4): 1
First cylinder (1-14593, default 1): 1
Last cylinder or +size or +sizeM or +sizeK (1-14593, default 14593):
Using default value 14593

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

root@server-tribeca:/etc# mkfs.ext3 /dev/sdb1
mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
7331840 inodes, 29304560 blocks
1465228 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
895 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 22 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

root@server-tribeca:/etc# ls /dev/disk/by-uuid -alh
lrwxrwxrwx 1 root root 10 2008-05-07 17:58 753a86d9-6228-4725-a9ae-6f457c2b1824 -> ../../sdb1
lrwxrwxrwx 1 root root 10 2008-05-07 17:58 dbd44197-c1c5-427d-9a34-40f652b92f42 -> ../../sda1
lrwxrwxrwx 1 root root 10 2008-05-07 17:58 e15f5c2c-fc1f-414d-8c67-99c567f44d6a -> ../../sda5

root@server-tribeca:/etc# mkdir /disk1

root@server-tribeca:/etc# vi /etc/fstab
# /etc/fstab: static file system information.
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
# /dev/sda1
UUID=dbd44197-c1c5-427d-9a34-40f652b92f42 / ext3 relatime,errors=remount-ro 0 1
# /dev/sda5
UUID=e15f5c2c-fc1f-414d-8c67-99c567f44d6a none swap sw 0 0
/dev/scd0 /media/cdrom0 udf,iso9660 user,noauto,exec,utf8 0 0
/dev/fd0 /media/floppy0 auto rw,user,noauto,exec,utf8 0 0
UUID=753a86d9-6228-4725-a9ae-6f457c2b1824 /disk1 ext3 defaults,errors=remount-ro 0 1

Changing Timezon and NTP On Ubuntu server

Posted in System with tags , , , on May 5, 2008 by wsjoung

Timezon Config

dpkg-reconfigure tzdata

Time Synchronisation with NTP

as root, create a file /etc/cron.daily/ntpdate containing:
The file /etc/cron.daily/ntpdate must also be executable
sudo chmod 755 /etc/cron.daily/ntpdate

pure-ftpd-mysql, [ERROR] Home directory not available – aborting

Posted in System with tags , , , , on April 15, 2008 by wsjoung

groupadd -g 2001 ftpgroup
useradd -u 2001 -s /bin/false -d /bin/null -c “pureftpd user” -g ftpgroup ftpuser

Instead of this dummy ftpuser account setting when they want to create real linux account.
they may get this error “Home directory not available – abort” even if they set CreateHomeDir to yes.

the last existing in the home path should be own by root.
for example, we want to create somebody’s home directory when he log in, /home/ftpuser/somebody
then the last existing directory ‘ftpuer’ may look like this,
drwxr-xr-x 2 root ftpgroup 4096 2008-04-15 11:20 ftpuser

[pure-ftpd] CreateHomeDir problem (“[ERROR] Home directory not available – aborting”)
Virtual Hosting With PureFTPd And MySQL (Incl. Quota And Bandwidth Management) On Ubuntu 7.10 (Gutsy Gibbon)

Linux signal

Posted in System with tags , , , on November 17, 2006 by wsjoung

I was writing a shell scripts to manage my process. basically I have some jobs need to be run for long time at least 2 weeks, but the problem is that the system manager will kill all my processes during the night because it takes too much cpu resource and it increases the temperature.

So, I wanted to write a small scripts which can make all my processes stop during the night and resume next day.

There is SIGNALs

`HUP’ : 1. Hangup.

`INT’ : 2. Terminal interrupt.

`QUIT’ : 3. Terminal quit.

`ABRT’ : 6. Process abort.

`KILL’ : 9. Kill (cannot be caught or ignored).

`ALRM’ : 14. Alarm Clock.

`TERM’ : 15. Termination.

`BUS’ : Access to an undefined portion of a memory object.

`CHLD’ : Child process terminated, stopped, or continued.

`CONT’ : Continue executing, if stopped.

`FPE’ : Erroneous arithmetic operation.

`ILL’ : Illegal Instruction.

`PIPE’ : Write on a pipe with no one to read it.

`SEGV’ : Invalid memory reference.

`STOP’ : Stop executing (cannot be caught or ignored).

`TSTP’ : Terminal stop.

`TTIN’ : Background process attempting read.

`TTOU’ : Background process attempting write.

`URG’ : High bandwidth data is available at a socket.

Especially TSTP and CONT was useful for my job.
for example,

for line in $(cat $hosts)
pid=$(rsh $line ps aux|grep “rosetta.gcc”|awk ‘{print $2}’);
if [ $pid ] ; then
rsh $line kill -TSTP $pid

The Microarchitecture of the Pentium 4 Processor

Posted in System with tags , , on November 17, 2006 by wsjoung

It has been a while since the Pentium 4 Processor came out though, it is one of the most successful processors. so I believe that it still has a worth to read.

What is the topic of this paper?
The microarchitecture of the pentium 4 processor

What is the problem being addressed by the work reported on in the paper?
The computing environment needs more power and many applications need to process multi-media data like MPEG2/4 streaming video or audio.

Why is this problem important?
General improvement of the architecture is not enough to follow up the exponentially growing request of the computing power which caused by multi-media data processing. and successful architecture which is specialized in muti-media might be good chance for home computing market.

What is the research’s basic approach?
NetBusrst mocroarchitecture
Intel introduced NetBusrst mocroarchitecture to deal with balancing and tuning of many mocroarchitecture features that compete for processor die cost and for design and validation efforts in a fast processor.
This NetBusrst mocroarchitecture is composed of 4 sections.

– Front End
In Front End, instructions are fetched to uops and stored and executed from trace cache, only when there is a trace cache miss instructions are fetched and decoded from the L2 cache. it also has guided by branch predictor that directs where instruction fetching needs to go next in the trace cache so that they could reduced misprediction rate. and because of prefetcher, L2 cache maintains 256 bytes ahead of the current dfata access locations.

– Out-of-Order Execution Logic
This part of the machine re-orders instructions to allow them to execute as quickly as their input operands are ready by allocation buffers, register renaming, and uop scheduling functions.

– Integer and Floating-Point Execution Units
Execution units are the place where the instructions are actually executed. and there are several different execution units. there is a separate 128 entry register file for both the integer and the floating-point/SSE(Streaming SIMD Extension) operations. and each register file also has a multi-clock bypass network that bypasses or forwards just completed results.

– Memory Subsystem
The Memory Subsystem includes the L2 cache and the system bus. L2 cache is a cache for L1, trace cache and the system bus is a IO channel for memory. memory subsystem has high bandwidth, – L2 cache is 48Gbytes per sec when running at 1.5GHz, system bus is 3.2Gbytes per sec- it is critical to processing streaming data from memory.

Is this approach successful?
yes, the performance comparison graph showed 15-20% (interger) and 30-70% (floating point and multi-media) higher performance to compare with pentium 3 processor.

Where should this research go next?
Intel should move to 64 bit architecture next with better caching algorithm.
For better understanding, check those links.

Samba Setting: Secure Read-Write File and Print Server

Posted in System with tags , , , on November 17, 2006 by wsjoung

Secure Read-Write File and Print Server

We progress now from simple systems to a server that is slightly more complex.

Our new server will require a public data storage area in which only authenticated users (i.e., those with a local account) can store files, as well as a home directory. There will be one printer that should be available for everyone to use.

In this hypothetical environment (no espionage was conducted to obtain this data), the site is demanding a simple environment that is secure enough but not too difficult to use.

Site users will be Jack Baumbach, Mary Orville, and Amed Sehkah. Each will have a password (not shown in further examples). Mary will be the printer administrator and will own all files in the public share.

This configuration will be based on user-level security that is the default, and for which the default is to store Microsoft Windows-compatible encrypted passwords in a file called /etc/samba/smbpasswd. The default smb.conf entry that makes this happen is passdb backend = smbpasswd, guest. Since this is the default, it is not necessary to enter it into the configuration file. Note that the guest backend is added to the list of active passdb backends no matter whether it specified directly in Samba configuration file or not.

Procedure 2.2. Installing the Secure Office Server

Example 2.4. Secure Office Server smb.conf
# Global parameters
workgroup = MIDEARTH
netbios name = OLORIN
printcap name = cups
disable spoolss = Yes
show add printer wizard = No
printing = cups
comment = Home Directories
valid users = %S
read only = No
browseable = No
comment = Data
path = /export
force user = maryo
force group = users
guest ok = Yes
read only = No
comment = All Printers
path = /var/spool/samba
printer admin = root, maryo
create mask = 0600
guest ok = Yes
printable = Yes
use client driver = Yes
browseable = No

1. Add all users to the operating system:
root# useradd -c “Jack Baumbach” -m -g users -p m0r3pa1n jackb
root# useradd -c “Mary Orville” -m -g users -p secret maryo
root# useradd -c “Amed Sehkah” -m -g users -p secret ameds

2. Configure the Samba smb.conf file as shown above

3. Initialize the Microsoft Windows password database with the new users:
root# smbpasswd -a root
New SMB password: bigsecret
Reenter smb password: bigsecret
Added user root.

root# smbpasswd -a jackb
New SMB password: m0r3pa1n
Retype new SMB password: m0r3pa1n
Added user jackb.

root# smbpasswd -a maryo
New SMB password: secret
Reenter smb password: secret
Added user maryo.

root# smbpasswd -a ameds
New SMB password: mysecret
Reenter smb password: mysecret
Added user ameds.

4. Install printer using the CUPS Web interface. Make certain that all printers that will be shared with Microsoft Windows clients are installed as raw printing devices.

5. Start Samba using the operating system administrative interface. Alternately, this can be done manually by executing:

root# nmbd; smbd;

Both applications automatically execute as daemons. Those who are paranoid about maintaining control can add the -D flag to coerce them to start up in daemon mode.

6. Configure the /export directory:
root# mkdir /export
root# chown maryo.users /export
root# chmod u=rwx,g=rwx,o-rwx /export

7. Check that Samba is running correctly:
root# smbclient -L localhost -U%

The following error message indicates that Samba was not running:
Error connecting to (Connection refused)
Connection to olorin failed

8. Connect to OLORIN as maryo:
root# smbclient //olorin/maryo -Umaryo%secret
OS=[UNIX] Server=[Samba-3.0.20]
smb: \> dir
smb: \> q

By now you should be getting the hang of configuration basics. Clearly, it is time to explore slightly more complex examples. For the remainder of this chapter we abbreviate instructions, since there are previous examples.

The Official Samba-3 HOWTO and Reference Guide