2009/06/23
2009/06/20
2009/06/10
2009/06/06
SLES10.2のxen3.2でbondingを利用する方法
Hassle-free Xen Networking
よく分からないけど、次のステップでいけるそうな。
1.通常のカーネルで、以下の３つのファイルを作成する。（アドレスは任意で）
それと同時に、routeとresolve.confの設定を行う。
---original "/etc/sysconfig/network/ifcfg-bond0" --
BONDING_MASTER='yes'
BONDING_MODULE_OPTS='mode=balance-rr miimon=100'
BONDING_SLAVE0='eth-id-00:e0:81:32:87:88'
BONDING_SLAVE1='eth-id-00:e0:81:32:87:89'
BOOTPROTO='dhcp'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
----------------------------------------------------
---adapted "/etc/sysconfig/network/ifcfg-bond0" --
BONDING_MASTER='yes'
BONDING_MODULE_OPTS='mode=balance-rr miimon=100'
BONDING_SLAVE0='eth-id-00:e0:81:32:87:88'
BONDING_SLAVE1='eth-id-00:e0:81:32:87:89'
BOOTPROTO='none'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK='
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
---------------------------------------------------
----new "/etc/sysconfig/network/ifcfg-br0"---------
BOOTPROTO='dhcp'
BRIDGE='yes'
BRIDGE_PORTS='bond0'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
---------------------------------------------------
2./etc/xen/xend-config.sxpを以下のように変更し、xenカーネルを指定し再起動する。
#(network-script network-bridge)
(network-script)
domainUの設定ファイルのNW設定を以下のようにする。
vif=[ 'mac=00:16:3e:0a:15:6c,bridge=br0', ]
vif=[ 'mac=00:16:3e:7e:f3:c7,model=ne2k_pci,type=ioemu,bridge=br0', ]
vif=[ 'mac=00:16:3e:7c:6b:f3,bridge=br0', ]
以上
***以下、関連ソース***
This document (7000616) is provided subject to the disclaimer at the end of this document.
Environment
Novell SUSE Linux Enterprise Server 10 Service Pack 2
Situation
Many problems arise from the use of the network-bridge script from SLES 10 SP2.
For example the following might occur:
* The network-bridge script does not work for multiple xen bridges
* The network-bridge script does not work very well with dhcp enabled devices
* The network-bridge script does not transfer additional routes to xen bridge
* You want to get rid of the "useless" P-devices and have a consistent network setup even when booting between normal and xen kernel.
Resolution
We can't really provide a single step by step solution. Instead we want to describe how it works.
First we recommend to boot into the normal (not xen) kernel and configure all network devices as you like to have a working setup.
Now for each xenbridge you later want to use create a configuration file for a normal linux bridge device that can be handled well by ifup/ifdown scripts. Linux bridges are identified by the ifcfg filename
ifcfg-br
The following things are important:
* insert your original device name to the option BRIDGE_PORTS of the bridge configuration file
* transfer the ip configuration from the original device to the bridge device (BOOTPROTO,IPADDR,NETMASK etc.)
* in the original device configuration set the BOOTPROTO to "none"
Here is an example:
The original network device is bond0 and we want to create the xenbridge as br0.
---original "/etc/sysconfig/network/ifcfg-bond0" --
BONDING_MASTER='yes'
BONDING_MODULE_OPTS='mode=balance-rr miimon=100'
BONDING_SLAVE0='eth-id-00:e0:81:32:87:88'
BONDING_SLAVE1='eth-id-00:e0:81:32:87:89'
BOOTPROTO='dhcp'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
----------------------------------------------------
---adapted "/etc/sysconfig/network/ifcfg-bond0" --
BONDING_MASTER='yes'
BONDING_MODULE_OPTS='mode=balance-rr miimon=100'
BONDING_SLAVE0='eth-id-00:e0:81:32:87:88'
BONDING_SLAVE1='eth-id-00:e0:81:32:87:89'
BOOTPROTO='none'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK='
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
---------------------------------------------------
----new "/etc/sysconfig/network/ifcfg-br0"---------
BOOTPROTO='dhcp'
BRIDGE='yes'
BRIDGE_PORTS='bond0'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
---------------------------------------------------
If you have static IP addresses with static routing configuration, you might also want to adjust the file /etc/sysconfig/networks/routes and insert the bridgename instead of the original device or the - wild-card for any device. (This prevents warnings about the routing configuration during network start)
before:
192.168.23.0 192.168.42.23 255.255.255.0 - bond0
default 192.168.42.23 - -
after:
192.168.23.0 192.168.42.23 255.255.255.0 - br0
default 192.168.42.23 - br0
That is all you need to do, now your network will work in the normal and the xen environment. The next step is to disable the network-bridge script. To do this, open the file /etc/xen/xend-config.sxp into your favorite editor, search for any line similar to
(network-script network-bridge)
and comment it out:
# (network-script network-bridge)
As final step (not necessary if you only use one bridge and never configured xen to use a bridge by name) you may need to adapt any existing guest configuration to use the new bridge, instead ofxenbr. Open each guest configuration file (they are located in directory /etc/xen/vm/ and search for a line similar to
vif=[ 'mac=00:16:3e:0a:15:6c,bridge=xenbr0', ]
vif=[ 'mac=00:16:3e:7e:f3:c7,model=ne2k_pci,type=ioemu', ]
vif=[ 'mac=00:16:3e:7c:6b:f3', ]
and replace it with:
vif=[ 'mac=00:16:3e:0a:15:6c,bridge=br0', ]
vif=[ 'mac=00:16:3e:7e:f3:c7,model=ne2k_pci,type=ioemu,bridge=br0', ]
vif=[ 'mac=00:16:3e:7c:6b:f3,bridge=br0', ]
This was the last step. Now you can boot again into the xen kernel and everything should work as expected.
Note: If you want to create a new guest via virt-manager later, make sure you edit the networking adapter option and replace the source from default to your bridge (like br0). This is only needed if you have more than one bridge.
Note: If you are using the SuSEFirewall2 to secure your Xen Host and Guests you may also need to adapt the /etc/sysconfig/SuSEfirewall2 to use your bridge (like br0) instead of xenbr0.
Tip: If you like to use YaST2 to configure the networking and you use a bonding device you have to set the Device Activation to on boot time and assign a static ip address 0.0.0.0 and Subnet Mask 255.255.255.255 for the bonding device, else the configuration dialog for the bridge does not show up any selectable interface.
Update 07.Feb.2009:
When using arp-monitoring for bonding it is essential that the bonding device has an ip address assigned. When the bonding device has no ip address you will notice slave flapping in the logfiles. To not confuse the routing it is also needed that no route supersedes the route from the bridge device. To assign a ip address to the bonding device please use following solution.
Create a script cp_ip_br_to_bond in /etc/sysconfig/network/scripts with following content:
------cp_ip_br_to_bond--------------
#!/bin/bash
# get bonding device
. /etc/sysconfig/network/ifcfg-$1
# cp ip from bridge to bonding device
ip addr show dev $2 | egrep '^ *inet ' | sed -e "
s/inet/ip addr add/
s@\([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+/[0-9]\+\)@\1@
s/$2/dev ${BRIDGE_PORTS} label ${BRIDGE_PORTS}/
s/secondary//
" | sh -e
# Remove automatic routes on bonding device
ip route list | sed -ne "
/dev ${BRIDGE_PORTS}\( \|$\)/ {
s/^/ip route del /
p
}" | sh -e
--------------------------------------------------
make the script executable with
chmod +x /etc/sysconfig/network/scripts/cp_ip_br_to_bond
and include it in the ifcfg-br0 file as
POST_UP_SCRIPT='cp_ip_br_to_bond'
We are still working on a final fix inside the ifup/ifdown scripts without the need of this workaround. We will post an update as soon we have released a final fix.
Update 01.Aug.2008:
- fixed device order in static routing example
- you might also want to set BRIDGE_FORWARDDELAY="0" in your /etc/sysconfig/ifcfg-br configuration file. This disables the default forwarding delay (15 sec.) and the interface is faster ready to send / receive.
Status
Top Issue
Document
Document ID: 7000616
Creation Date: 06-06-2008
Modified Date: 05-11-2009
Novell Product: SUSE Linux Enterprise Server
Disclaimer
The Origin of this information may be internal or external to Novell. Novell makes all reasonable efforts to verify this information. However, the information provided in this document is for your information only. Novell makes no explicit or implied claims to the validity of this information.
Any trademarks referenced in this document are the property of their respective owners. Consult your product manuals for complete trademark information.
よく分からないけど、次のステップでいけるそうな。
1.通常のカーネルで、以下の３つのファイルを作成する。（アドレスは任意で）
それと同時に、routeとresolve.confの設定を行う。
---original "/etc/sysconfig/network/ifcfg-bond0" --
BONDING_MASTER='yes'
BONDING_MODULE_OPTS='mode=balance-rr miimon=100'
BONDING_SLAVE0='eth-id-00:e0:81:32:87:88'
BONDING_SLAVE1='eth-id-00:e0:81:32:87:89'
BOOTPROTO='dhcp'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
----------------------------------------------------
---adapted "/etc/sysconfig/network/ifcfg-bond0" --
BONDING_MASTER='yes'
BONDING_MODULE_OPTS='mode=balance-rr miimon=100'
BONDING_SLAVE0='eth-id-00:e0:81:32:87:88'
BONDING_SLAVE1='eth-id-00:e0:81:32:87:89'
BOOTPROTO='none'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK='
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
---------------------------------------------------
----new "/etc/sysconfig/network/ifcfg-br0"---------
BOOTPROTO='dhcp'
BRIDGE='yes'
BRIDGE_PORTS='bond0'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
---------------------------------------------------
2./etc/xen/xend-config.sxpを以下のように変更し、xenカーネルを指定し再起動する。
#(network-script network-bridge)
(network-script)
domainUの設定ファイルのNW設定を以下のようにする。
vif=[ 'mac=00:16:3e:0a:15:6c,bridge=br0', ]
vif=[ 'mac=00:16:3e:7e:f3:c7,model=ne2k_pci,type=ioemu,bridge=br0', ]
vif=[ 'mac=00:16:3e:7c:6b:f3,bridge=br0', ]
以上
***以下、関連ソース***
This document (7000616) is provided subject to the disclaimer at the end of this document.
Environment
Novell SUSE Linux Enterprise Server 10 Service Pack 2
Situation
Many problems arise from the use of the network-bridge script from SLES 10 SP2.
For example the following might occur:
* The network-bridge script does not work for multiple xen bridges
* The network-bridge script does not work very well with dhcp enabled devices
* The network-bridge script does not transfer additional routes to xen bridge
* You want to get rid of the "useless" P-devices and have a consistent network setup even when booting between normal and xen kernel.
Resolution
We can't really provide a single step by step solution. Instead we want to describe how it works.
First we recommend to boot into the normal (not xen) kernel and configure all network devices as you like to have a working setup.
Now for each xenbridge you later want to use create a configuration file for a normal linux bridge device that can be handled well by ifup/ifdown scripts. Linux bridges are identified by the ifcfg filename
ifcfg-br
The following things are important:
* insert your original device name to the option BRIDGE_PORTS of the bridge configuration file
* transfer the ip configuration from the original device to the bridge device (BOOTPROTO,IPADDR,NETMASK etc.)
* in the original device configuration set the BOOTPROTO to "none"
Here is an example:
The original network device is bond0 and we want to create the xenbridge as br0.
---original "/etc/sysconfig/network/ifcfg-bond0" --
BONDING_MASTER='yes'
BONDING_MODULE_OPTS='mode=balance-rr miimon=100'
BONDING_SLAVE0='eth-id-00:e0:81:32:87:88'
BONDING_SLAVE1='eth-id-00:e0:81:32:87:89'
BOOTPROTO='dhcp'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
----------------------------------------------------
---adapted "/etc/sysconfig/network/ifcfg-bond0" --
BONDING_MASTER='yes'
BONDING_MODULE_OPTS='mode=balance-rr miimon=100'
BONDING_SLAVE0='eth-id-00:e0:81:32:87:88'
BONDING_SLAVE1='eth-id-00:e0:81:32:87:89'
BOOTPROTO='none'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK='
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
---------------------------------------------------
----new "/etc/sysconfig/network/ifcfg-br0"---------
BOOTPROTO='dhcp'
BRIDGE='yes'
BRIDGE_PORTS='bond0'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
---------------------------------------------------
If you have static IP addresses with static routing configuration, you might also want to adjust the file /etc/sysconfig/networks/routes and insert the bridgename instead of the original device or the - wild-card for any device. (This prevents warnings about the routing configuration during network start)
before:
192.168.23.0 192.168.42.23 255.255.255.0 - bond0
default 192.168.42.23 - -
after:
192.168.23.0 192.168.42.23 255.255.255.0 - br0
default 192.168.42.23 - br0
That is all you need to do, now your network will work in the normal and the xen environment. The next step is to disable the network-bridge script. To do this, open the file /etc/xen/xend-config.sxp into your favorite editor, search for any line similar to
(network-script network-bridge)
and comment it out:
# (network-script network-bridge)
As final step (not necessary if you only use one bridge and never configured xen to use a bridge by name) you may need to adapt any existing guest configuration to use the new bridge, instead ofxenbr. Open each guest configuration file (they are located in directory /etc/xen/vm/ and search for a line similar to
vif=[ 'mac=00:16:3e:0a:15:6c,bridge=xenbr0', ]
vif=[ 'mac=00:16:3e:7e:f3:c7,model=ne2k_pci,type=ioemu', ]
vif=[ 'mac=00:16:3e:7c:6b:f3', ]
and replace it with:
vif=[ 'mac=00:16:3e:0a:15:6c,bridge=br0', ]
vif=[ 'mac=00:16:3e:7e:f3:c7,model=ne2k_pci,type=ioemu,bridge=br0', ]
vif=[ 'mac=00:16:3e:7c:6b:f3,bridge=br0', ]
This was the last step. Now you can boot again into the xen kernel and everything should work as expected.
Note: If you want to create a new guest via virt-manager later, make sure you edit the networking adapter option and replace the source from default to your bridge (like br0). This is only needed if you have more than one bridge.
Note: If you are using the SuSEFirewall2 to secure your Xen Host and Guests you may also need to adapt the /etc/sysconfig/SuSEfirewall2 to use your bridge (like br0) instead of xenbr0.
Tip: If you like to use YaST2 to configure the networking and you use a bonding device you have to set the Device Activation to on boot time and assign a static ip address 0.0.0.0 and Subnet Mask 255.255.255.255 for the bonding device, else the configuration dialog for the bridge does not show up any selectable interface.
Update 07.Feb.2009:
When using arp-monitoring for bonding it is essential that the bonding device has an ip address assigned. When the bonding device has no ip address you will notice slave flapping in the logfiles. To not confuse the routing it is also needed that no route supersedes the route from the bridge device. To assign a ip address to the bonding device please use following solution.
Create a script cp_ip_br_to_bond in /etc/sysconfig/network/scripts with following content:
------cp_ip_br_to_bond--------------
#!/bin/bash
# get bonding device
. /etc/sysconfig/network/ifcfg-$1
# cp ip from bridge to bonding device
ip addr show dev $2 | egrep '^ *inet ' | sed -e "
s/inet/ip addr add/
s@\([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+/[0-9]\+\)@\1@
s/$2/dev ${BRIDGE_PORTS} label ${BRIDGE_PORTS}/
s/secondary//
" | sh -e
# Remove automatic routes on bonding device
ip route list | sed -ne "
/dev ${BRIDGE_PORTS}\( \|$\)/ {
s/^/ip route del /
p
}" | sh -e
--------------------------------------------------
make the script executable with
chmod +x /etc/sysconfig/network/scripts/cp_ip_br_to_bond
and include it in the ifcfg-br0 file as
POST_UP_SCRIPT='cp_ip_br_to_bond'
We are still working on a final fix inside the ifup/ifdown scripts without the need of this workaround. We will post an update as soon we have released a final fix.
Update 01.Aug.2008:
- fixed device order in static routing example
- you might also want to set BRIDGE_FORWARDDELAY="0" in your /etc/sysconfig/ifcfg-br configuration file. This disables the default forwarding delay (15 sec.) and the interface is faster ready to send / receive.
Status
Top Issue
Document
Document ID: 7000616
Creation Date: 06-06-2008
Modified Date: 05-11-2009
Novell Product: SUSE Linux Enterprise Server
Disclaimer
The Origin of this information may be internal or external to Novell. Novell makes all reasonable efforts to verify this information. However, the information provided in this document is for your information only. Novell makes no explicit or implied claims to the validity of this information.
Any trademarks referenced in this document are the property of their respective owners. Consult your product manuals for complete trademark information.
2009/06/05
登録:
投稿 (Atom)