Proxmox 3.4-6 IPV6 Networking - ARP Issues

We have just deploy PROXMOX and trying to use IPv6 Networking on VMs.

1.- In brand new Host (Proxmox) we add IPv6 addressing:

root@server:~# more /etc/network/interfaces

By default vmbr0 interface is edited (as it is being used as bridge to VMs)

auto vmbr0
iface vmbr0 inet static
 address  A.B.C.D        
 netmask  255.255.255.192        
 gateway  A.B.C.1        
 bridge_ports eth1        
 bridge_stp off        
 bridge_fd 0
iface vmbr0 inet6 static 
        address 2187:f0d2:3034:de::137
        netmask 64 
        gateway 2187:f0d2:3034:de::1
2.- Verify configuration update after a networking restart

root@server:~# ifconfig vmbr0 
vmbr0     Link encap:Ethernet  HWaddr 00:25:90:57:db:47           
inet addr:A.B.C.D  Bcast:A.B.C.191  Mask:255.255.255.192          
inet6 addr: 2187:f0d2:3034:de::137/64 Scope:Global          
inet6 addr: fe80::225:90ff:fe57:db47/64 Scope:Link          
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1          
RX packets:3239 errors:0 dropped:0 overruns:0 frame:0          
TX packets:1722 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 
RX bytes:803279 (784.4 KiB)   TX bytes:1530868 (1.4 MiB)

root@server:~# route -n6 
Kernel IPv6 routing tableDestination                    Next Hop                   Flag Met Ref Use If2187:f0d2:3034:de::/64         ::                         U    256 0     1 vmbr0 
fe80::1/128                    ::                         U    256 0     0 venet0 
fe80::/64                      ::                         U    256 0     0 vmbr0 
fe80::/64                      ::                         U    256 0     0 eth0 
fe80::/64                      ::                         U    256 0     0 eth1 
fe80::/64                      ::                         U    256 0     0 venet0 
::/0                           2187:f0d2:3034:de::1       UG   1024 0    18 vmbr0 
::/0                           ::                         !n   -1  1    26 lo 
::1/128                        ::                         Un   0   1     4 lo 
2187:f0d2:3034:de::137/128     ::                         Un   0   1    39 lo 
fe80::1/128                    ::                         Un   0   1     0 lo 
fe80::225:90ff:fe57:db46/128   ::                         Un   0   1     0 lo 
fe80::225:90ff:fe57:db47/128   ::                         Un   0   1     0 lo 
fe80::225:90ff:fe57:db47/128   ::                         Un   0   1     0 lo 
ff00::/8                       ::                         U    256 0     0 vmbr0 
ff00::/8                       ::                         U    256 0     0 eth0 
ff00::/8                       ::                         U    256 0     0 eth1 
ff00::/8                       ::                         U    256 0     0 venet0 
::/0                           ::                         !n   -1  1    26 lo

3.- Try some ping connectivity to GW. Of course maybe you will receive no response.
root@server:~# ping6  2187:f0d2:3034:de::1 
PING 2187:f0d2:3034:de::1(2607:f0d0:3001:de::1) 56 data bytes 
From 2187:f0d2:3034:de::137 icmp_seq=1 Destination unreachable: Address unreachable 
From 2187:f0d2:3034:de::137 icmp_seq=3 Destination unreachable: Address unreachable 
From 2187:f0d2:3034:de::137 icmp_seq=4 Destination unreachable: Address unreachable 
From 2187:f0d2:3034:de::137 icmp_seq=5 Destination unreachable: Address unreachable 
^C 
--- 2187:f0d2:3034:de::1 ping statistics ---6 packets transmitted, 0 received, +4 errors, 100% packet loss, time 5000ms
root@server:~# ping6 ipv6.google.com 
PING ipv6.google.com(iad23s43-in-x0e.1e100.net) 56 data bytes 
From 2187:f0d2:3034:de::137 icmp_seq=1 Destination unreachable: Address unreachable 
From 2187:f0d2:3034:de::137 icmp_seq=3 Destination unreachable: Address unreachable 
From 2187:f0d2:3034:de::137 icmp_seq=4 Destination unreachable: Address unreachable 
From 2187:f0d2:3034:de::137 icmp_seq=5 Destination unreachable: Address unreachable 
^C--- 
ipv6.google.com ping statistics ---7 packets transmitted, 0 received, +3 errors, 100% packet loss, time 6064ms

4.- Adding Loopback interface like this:

iface lo inet6 loopback
5.- Troubleshooting

Verify ARP table (ip -6 neigh)
root@server:~# ip neigh ls 
2187:f0d2:3034:de::1 dev vmbr0  FAILED 
A.B.C.1 dev vmbr0 lladdr 00:08:e3:ff:fd:90 REACHABLE
Showing that ARP resolution failed. So You have to verify properly configuration of IPv6 address on the other side (the router gateway).

I am discarding some physical connectivity issue as my router was resolved properly against its IPv4 address. (assigned - dual stack for GW)

You could also verify Table:

root@server:~# arp -na 
? (A.B.C.1) at 00:08:e3:ff:fd:90 [ether] on vmbr0
Again: Resolution for your GW over IPv6 failed.

6.- Force your server (temporal testing only)

If your NOC or Networking specialist confirmed you that IPv6 address is already assigned to GW try to force adding that entry into ARP table:

ip -6 neigh replace 2187:f0d2:3034:de::1 dev vmbr0 lladdr 00:08:e3:ff:fd:90

Then you could ping GW and ping6 everywhere.
If that worked for you... then you could add that permanent rule inside /etc/network/interface file

iface vmbr0 inet6 static 
    address 2187:f0d2:3034:de::137 
    netmask 64 
    gateway 2187:f0d2:3034:de::1 
    post-up sysctl -p 
    post-up ip -6 neigh replace 2187:f0d2:3034:de::1 dev vmbr0 lladdr 00:08:e3:ff:fd:90


PD: If you want to remove ARP entry you could use:
Review:
root@server:~# ip neigh ls 

2187:f0d2:3034:de::1 dev vmbr0 lladdr 00:08:e3:ff:fd:90 PERMANENT
 
A.B.C.1 dev vmbr0 lladdr 00:08:e3:ff:fd:90 REACHABLE 
 Delete
root@server:~# ip -6 neigh del 2187:f0d2:3034:de::1 dev vmbr0 
Verify:
root@server:~# ip neigh ls 
2187:f0d2:3034:de::1 dev vmbr0  FAILED 
A.B.C.1 dev vmbr0 lladdr 00:08:e3:ff:fd:90 REACHABLE

UPDATE: Depending on your Proxmox Kernel, you will need to update it to discard ICMPv6 discovery issues.
More info is posted here

Comments