OSPF Troubleshooting 2 Lab

This lab was taken from gns3vault.com. Click here to go to the lab page.

Goal

All IP addresses have been preconfigured for you.
OSPF is preconfigured with the areas as as specified in the topology picture.
Do not use show run! use the appropiate 'show' and 'debug' commands.

  1. Make sure all OSPF neighbor adjacencies are working!

Topology

c5ospf2.png

Configuration

1

Let's begin with Nick in area 0.

Nick#show ip protocols
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 192.168.13.1
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    192.168.12.0 0.0.0.0 area 0
    192.168.13.0 0.0.0.255 area 0
 Reference bandwidth unit is 100 mbps
  Routing Information Sources:
    Gateway         Distance      Last Update
  Distance: (default is 110)

I appears that Nick is properly configured to form an adjacency with Mike (192.168.13.3), but not with Joey (192.168.12.2).
The first statement, "192.168.12.0 0.0.0.0 area 0", matches only and exactly the IP address 192.168.12.0, which does not belong to any of Nick's interfaces.

Nick(config)#router ospf 1
Nick(config-router)#no network 192.168.12.0 0.0.0.0 area 0
Nick(config-router)#network 192.168.12.1 0.0.0.0 area 0

The 'show ip ospf neighbor' command reveals only one attempt to form a neighbor relationship.

Nick#show ip ospf neighbor

Neighbor ID     Pri   State           Dead Time   Address         Interface
192.168.26.2      0   2WAY/DROTHER    00:00:33    192.168.12.2    FastEthernet0/0

That is with Joey, who inherited the 192.168.26.2 router-id due to it being the highest IP address on one of his interfaces.
They are stuck in two-way, however. This means that they see each other listed in the Hello packets. Two-way is also when DR-BDR election happens, which is most likely the cause of their relationship not being able to progress.

Nick#show ip ospf int f0/0
FastEthernet0/0 is up, line protocol is up
  Internet Address 192.168.12.1/24, Area 0
  Process ID 1, Router ID 192.168.13.1, Network Type BROADCAST, Cost: 1
  Transmit Delay is 1 sec, State DROTHER, Priority 0

Joey#show ip ospf int f0/0
FastEthernet0/0 is up, line protocol is up
  Internet Address 192.168.12.2/24, Area 0
  Process ID 1, Router ID 192.168.26.2, Network Type BROADCAST, Cost: 1
  Transmit Delay is 1 sec, State DROTHER, Priority 0

They are both DROTHER, so they are not able to form a 'greater' relationship. The reason for this is that their priority is set to zero, as shown by the 'show ip ospf interface' commands, which does not allow them to become a candidate for the DR/BDR election.

It acts as a point-to-point connection so it is not relevant which one of them becomes the DR/BDR.

Nick(config)#int f0/0
Nick(config-if)#no ip ospf priority
Nick#
*Mar  1 00:17:33.683: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.26.2 on FastEthernet0/0 from LOADING to FULL, Loading Done

The above command resets Nick's OSPF priority on the interface to the default of 1, which allows it to become the DR. The OSPF neighbor relationship between Nick and Joey is formed.

Now, what is wrong with Mike? Nick did not even attempt to form a relationship with Mike.

Mike#show ip int br
Interface                  IP-Address      OK? Method Status                Protocol
FastEthernet0/0            192.168.34.3    YES NVRAM  up                    up
FastEthernet1/0            192.168.13.3    YES NVRAM  up                    up

Mike#show ip protocols
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 192.168.34.3
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    192.168.13.0 0.0.0.255 area 0
    192.168.34.0 0.0.0.255 area 0
 Reference bandwidth unit is 100 mbps
  Passive Interface(s):
    FastEthernet1/0

Mike has the network statement required to activate the interface to Nick, however the interface is set to passive. This means that no OSPF packets may be sent out that interface.

Mike(config)#router ospf 1
Mike(config-router)#no passive f1/0
*Mar  1 00:22:34.327: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.13.1 on FastEthernet1/0 from LOADING to FULL, Loading Done

Mike#show ip ospf neighbor

Neighbor ID     Pri   State           Dead Time   Address         Interface
192.168.13.1      1   FULL/DR         00:00:34    192.168.13.1    FastEthernet1/0

Mike is now up as well, but he also has neighbor relationship problems with Angie.

Angie#show ip protocols
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 192.168.45.4
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    192.168.24.0 0.0.0.255 area 0
    192.168.34.0 0.0.0.255 area 0
    192.168.45.0 0.0.0.255 area 0

The above command shows that Angie's interface is enabled and we already know Mike's is up from when we checked the network statements early on for Nick.

Mike#show ip ospf int f0/0
FastEthernet0/0 is up, line protocol is up
  Internet Address 192.168.34.3/24, Area 0
  Process ID 1, Router ID 192.168.34.3, Network Type BROADCAST, Cost: 1
  Transmit Delay is 1 sec, State DR, Priority 1
  Designated Router (ID) 192.168.34.3, Interface address 192.168.34.3
  No backup designated router on this network
  Timer intervals configured, Hello 8, Dead 12, Wait 12, Retransmit 5

Angie#show ip ospf int f1/0
FastEthernet1/0 is up, line protocol is up
  Internet Address 192.168.34.4/24, Area 0
  Process ID 1, Router ID 192.168.45.4, Network Type BROADCAST, Cost: 1
  Transmit Delay is 1 sec, State DR, Priority 1
  Designated Router (ID) 192.168.45.4, Interface address 192.168.34.4
  No backup designated router on this network
  Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5

They both believe they are DR's and their timers do not match.

Mike(config)#int f0/0
Mike(config-if)#no ip ospf hello-interval
Mike(config-if)#no ip ospf dead-interval

After this change everything should match, but the relationship does not form.

Checking more IP information about the interface, I came across this piece of information:

Angie#show ip int f1/0
FastEthernet1/0 is up, line protocol is up
  Internet address is 192.168.34.4/24
  Broadcast address is 255.255.255.255
  Address determined by non-volatile memory
  MTU is 1500 bytes
  Helper address is not set
  Directed broadcast forwarding is disabled
  Multicast reserved groups joined: 224.0.0.5 224.0.0.6
  Outgoing access list is not set
  Inbound  access list is DEFEND
[...]

Interface f1/0 on Angie has an inbound access-list named DEFEND which is probably blocking traffic.

Angie#show ip access-l
Extended IP access list DEFEND
    10 deny ip any 224.0.0.0 0.255.255.255 (135 matches)

And indeed it is. Not only is it blocking all multicast addresses (including 224.0.0.5 and 224.0.0.6 used by OSPF), but also everything else due to the implicit deny at the end of the access-list.

Angie#ping 192.168.34.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.34.3, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

Including ping replies from Mike.

Angie(config)#int f1/0
Angie(config-if)#no ip access-group DEFEND in
Angie(config-if)#
*Mar  1 00:21:07.623: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.34.3 on FastEthernet1/0 from LOADING to FULL, Loading Done

Mike now has FULL relationships with his neighbors.

Joey>show ip protocols
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 192.168.26.2
  It is an area border router
  Number of areas in this router is 2. 2 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    192.168.12.0 0.0.0.255 area 0
    192.168.24.0 0.0.0.255 area 0
    192.168.26.0 0.0.0.255 area 1
[...]

Angie#show ip protocols
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 192.168.45.4
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    192.168.24.0 0.0.0.255 area 0
    192.168.34.0 0.0.0.255 area 0
    192.168.45.0 0.0.0.255 area 0
[...]

The network statements are properly configured on both ends, so it is most likely an interface-related problem.

Joey#show ip ospf int f1/0
FastEthernet1/0 is up, line protocol is up
  Internet Address 192.168.24.2/24, Area 0
  Process ID 1, Router ID 192.168.26.2, Network Type BROADCAST, Cost: 1
  Transmit Delay is 1 sec, State DR, Priority 1
[...]

Angie#show ip ospf int f0/0
FastEthernet0/0 is up, line protocol is up
  Internet Address 192.168.24.4/23, Area 0
  Process ID 1, Router ID 192.168.45.4, Network Type BROADCAST, Cost: 1
  Transmit Delay is 1 sec, State DR, Priority 1
[...]

If both of them consider themselves DR, we can be sure that there is a mismatch of at least one parameter. In this case, the network mask is different.
Everything else is a /24 address space, so Angie is most likely configured wrong.

Angie(config)#int f0/0
Angie(config-if)#ip add 192.168.24.4 255.255.255.0
*Mar  1 00:27:14.947: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.26.2 on FastEthernet0/0 from LOADING to FULL, Loading Done

Area 0 is not properly configured. Let's check the connections between area 0 and area 1, starting with Angie.

Jack#
*Mar  1 00:29:15.911: %OSPF-4-ERRRCV: Received invalid packet: mismatch area ID, from backbone area must be virtual-link but not found from 192.168.45.4, FastEthernet0/0
Jack#show ip protocols
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 192.168.45.5
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    192.168.45.0 0.0.0.255 area 1
 Reference bandwidth unit is 100 mbps
  Routing Information Sources:
    Gateway         Distance      Last Update
  Distance: (default is 110)

Right off the bat, Jack complains about Hello packets received from Angie. Everything appears to be ok on Jack, so Angie is probably misconfigured.

Angie#show ip protocols
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 192.168.45.4
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    192.168.24.0 0.0.0.255 area 0
    192.168.34.0 0.0.0.255 area 0
    192.168.45.0 0.0.0.255 area 0

Angie says that her 192.168.45.4 interface belongs to area 0, which is not correct.

Angie(config)#router ospf 1
Angie(config-router)#no  network 192.168.45.0 0.0.0.255 area 0
Angie(config-router)# network 192.168.45.0 0.0.0.255 area 1
Angie(config-router)#
*Mar  1 00:31:52.979: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.45.5 on FastEthernet2/0 from LOADING to FULL, Loading Done

Finally, Joey and Janice. Since they are connected through a Frame-Relay cloud, the problem is most likely incorrect or non-existent neighbor statements. In a NBMA network, OSPF neighbors must be manually configured using the 'neighbor' command.

Joey#show ip protocols
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 192.168.26.2
  It is an area border router
  Number of areas in this router is 2. 2 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    192.168.12.0 0.0.0.255 area 0
    192.168.24.0 0.0.0.255 area 0
    192.168.26.0 0.0.0.255 area 1
[...]

Janice#show ip protocols
Routing Protocol is "ospf 1"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 192.168.26.6
  Number of areas in this router is 1. 0 normal 0 stub 1 nssa
  Maximum path: 4
  Routing for Networks:
    192.168.26.0 0.0.0.255 area 1
 Reference bandwidth unit is 100 mbps
  Routing Information Sources:
    Gateway         Distance      Last Update
  Distance: (default is 110)

The network statements are fine, but wow! Janice thinks her only area, area 1, is a NSSA, but Joey disagrees. An area mismatch is not acceptable when forming neighbor relationships.

Janice(config)#router ospf 1
Janice(config-router)#no area 1 nssa
Janice#show ip ospf int s0/0
Serial0/0 is up, line protocol is up
  Internet Address 192.168.26.6/24, Area 1
  Process ID 1, Router ID 192.168.26.6, Network Type BROADCAST, Cost: 64
  Transmit Delay is 1 sec, State WAITING, Priority 1
  No designated router on this network
  No backup designated router on this network
  Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
    oob-resync timeout 40
    Hello due in 00:00:04
    Wait time before Designated router selection 00:00:04
  Supports Link-local Signaling (LLS)
  Index 1/1, flood queue length 0
  Next 0x0(0)/0x0(0)
  Last flood scan length is 1, maximum is 5
  Last flood scan time is 4 msec, maximum is 4 msec
  Neighbor Count is 0, Adjacent neighbor count is 0
  Suppress hello for 0 neighbor(s)
  Simple password authentication enabled

Joey#show ip ospf int s2/0
Serial2/0 is up, line protocol is up
  Internet Address 192.168.26.2/24, Area 1
  Process ID 1, Router ID 192.168.26.2, Network Type BROADCAST, Cost: 64
  Transmit Delay is 1 sec, State WAITING, Priority 1
  No designated router on this network
  No backup designated router on this network
  Flush timer for old DR LSA due in 00:01:50
  Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
    oob-resync timeout 40
    Hello due in 00:00:04
    Wait time before Designated router selection 00:00:14
  Supports Link-local Signaling (LLS)
  Index 1/3, flood queue length 0
  Next 0x0(0)/0x0(0)
  Last flood scan length is 5, maximum is 5
  Last flood scan time is 0 msec, maximum is 4 msec
  Neighbor Count is 0, Adjacent neighbor count is 0
  Suppress hello for 0 neighbor(s)
  Simple password authentication enabled

The network type is BROADCAST, not NBMA, so manual neighbor statements are not required if the frame-relay mappings are correctly configured.

Also, note that simple password authentication is enabled on both ends. This should be checked a well when normal configurations appears to be correct, but OSPF does not work.

Janice#show frame map
Serial0/0 (up): ip 192.168.26.2 dlci 602(0x25A,0x94A0), static,
              CISCO, status defined, active

Joey#show frame map
Serial2/0 (up): ip 192.168.26.6 dlci 206(0xCE,0x30E0), static,
              CISCO, status defined, active

And they are not. The keyword 'broadcast' must be there, otherwise frame-relay will not forward broadcasts/multicasts correctly.

Joey(config)#int s2/0
Joey(config-if)#frame map ip 192.168.26.6 206 broad

Janice(config)#int s0/0
Janice(config-if)#frame map ip 192.168.26.2 602 broad

Now to check the authentication. This can be done via debug, for example:

Joey#debug ip ospf adj
*Mar  1 00:47:43.127: OSPF: Rcv pkt from 192.168.26.6, Serial2/0 : Mismatch Authentication Key - Clear Text

I am not sure how to check the authentication keys without cheating with 'show run'.

There are at least three possible options in this case.

First is checking the running config and correct the authentication keys.
Second is replacing the keys with new ones.
Third is removing authentication altogether.

In the spirit of including as many configuration commands as possible, I will disable the plain text authentication and replace it with message-digest authentication using new keys.

Joey(config-if)#no ip ospf authentication
Joey(config-if)#ip ospf authentication message-digest
Joey(config-if)#no ip ospf authentication-key
Joey(config-if)#ip ospf message-digest-key 1 md5 cisco

Janice(config)#int s0/0
Janice(config-if)#no ip ospf authentication
Janice(config-if)#ip ospf authentication message-digest
Janice(config-if)#no ip ospf authentication-key
Janice(config-if)#ip ospf message-digest-key 1 md5 cisco
*Mar  1 00:58:05.791: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.26.2 on Serial0/0 from LOADING to FULL, Loading Done

The neighbor relationship between Joey and Janice is now formed. As a test, I tried a ping from Janice to Mike and it was successful:

Janice#ping 192.168.13.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.13.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 84/97/124 ms
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License