In this post I will discuss about the issue faced today while joing AP to WLC version 8.0.100.0.
5 Day before I got a new 2602 AP and Today I tried to connect to my switch in right AP VLAN. I saw that AP got IP address from DHCP pool and WLC IP via DHCP Option 43 and AP start updating the Image from WLC.
I was relaxed that it is working so I will test my Important topic like Auto Anchor, Static IP tunneling & Foreign mapping.
After 1-2 minutes I saw that there was some kind of failure which I never seen, here are the logs:
TestAP# *Nov 19 13:37:59.999: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS. *Nov 19 13:38:00.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.10.3 peer_port: 5246 *Nov 19 13:38:29.999: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_connection_db.c:2214 Max retransmission count reached for Connection 0x8D69EB4! *Nov 19 13:38:59.999: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 192.168.10.3:5246 *Nov 19 13:38:59.999: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS. *Nov 19 13:39:00.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.10.1 peer_port: 5246Peer certificate verification failed FFFFFFFF *Nov 19 13:39:00.099: DTLS_CLIENT_ERROR: ../capwap/base_capwap/capwap/base_capwap_wtp_dtls.c:496 Certificate verified failed! *Nov 19 13:39:00.099: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to 192.168.10.1:5246 *Nov 19 13:39:00.099: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 192.168.10.1:5246 TestAP#
After googling I got this: APs mfg in September/October 2014 unable to join an AireOS controller CSCur43050
Description
Symptom:
New Aironet APs with factory installed recovery IOS are able to join the controller 8.0.100.0 and download 15.3(3)JA IOS. But after the AP reload, the APs are unable to join the controller. On the AP, logs similar to the following are seen:
*Oct 16 12:39:06.231: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS. *Oct 16 13:14:56.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: ***.***.***.*** peer_port: 5246Peer certificate verification failed FFFFFFFF *Oct 16 13:14:56.127: DTLS_CLIENT_ERROR: ../capwap/base_capwap/capwap/base_capwap_wtp_dtls.c:496 Certificate verified failed! *Oct 16 13:14:56.127: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to ***.***.***.***:5246 *Oct 16 13:14:56.127: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to ***.***.***.***:5246
Another symptom of this problem is that the AP may be able to join the 8.0.100.0 controller, download the IOS code, boot up and join the controller OK … but when it goes to upgrade to newer 8.x code, it gets stuck in a loop failing the download.
Conditions:
Seen only with APs that were manufactured in September or October, 2014 – all Aironet APs were affected EXCEPT the 700 series. Seen with WLCs running 8.0.100.0 or an 8.0.100.x special.
If the WLC was manufactured in September 2014, or later (i.e. has a SHA2 MIC), then the first symptom is seen, i.e. the AP joins the 8.0.100 WLC, downloads the image, but then fails to rejoin.
If the WLC was manufactured before September 2014 (i.e. does not have a SHA2 MIC), then the second symptom is seen, i.e. the AP can join the 8.0.100 WLC OK, but then will fail download during a subsequent upgrade.
Also seen with new APs trying to join a controller running IOS-XE 3.6.0 (15.3(3)JN k9w8 image.) (Track CSCur50946 for the IOS-XE fix)
Workaround:
Downgrade to AireOS 7.6.130.0, or to IOS-XE 3.3, if the APs are supported in the earlier code.
Further Problem Description:
This problem affects only APs that were manufactured with incorrect SHA2
certificates. APs with only SHA1 certificates are not affected. To determine
whether an AP is affected, use the following AP exec commands (while the AP
has a 15.3(3)/8.0 image installed):
1. Check for the presence of a SHA2 Parameter Block:
ap#test pb display
if the output of this command includes:
SHA2 Parameter Block Doesn’t have any Records
then this AP is not affected. If the output of this command shows
Display of the SHA2 Parameter Block
then
2. See whether a correct SHA2 certificate is present:
ap#show crypto pki trustpoints | include SHA2
if there is no valid SHA2 certificate, then this will show no output.
If there is a valid SHA2 cert, this will show:
cn=Cisco Manufacturing CA SHA2
Only APs which *do* have a SHA2 Parameter Block and which *do not* have
a valid SHA2 certificate are affected by this bug.
The problem symptoms will vary according to whether or not the WLC has a
SHA2 certificate installed. To verify this, use the following command on
the AireOS CLI:
Cisco Controller) >show certificate all
and look for:
Certificate Name: Cisco SHA2 device cert
Then I downgraded my WLC to version 7.6.130.0 and it worked.
So this just a small post, it may help those who is/will get this kinda problem.
How long you expect to stay at 7.6?
Will stay with it till Cisco release WLC 8.1.
Regards
Thanks for sharing…
Think i ran into this a couple of weeks ago. Was in a hurry, so just use 7 ver.
Bg
Hi Kasper,
Great to see that you find this post useful.
Regards
Hey Rasika,
For unknown reason Cisco does not want to make life “easier” for some people. What I’m trying to say is: How can a user identify if the batch of newly delivered APs are or can be affected by this bug.
The easiest way is to determine by the serial number of the AP/APs. Serial numbers affected are the following:
fff1837xxxx up to fff1844xxxx
APs with serial numbers above are made between September 2014 up to the end of October 2014.
Hope this helps everyone reading this.
Best Regards/Leo
PS: I’m going to put a “fail” at the CBT about this Bug ID.
Hi LEO.
Thanks for additional info.
** its sandeep blog, Rasika blog is (mrncciew.com)
Regards
We bought 2 AIR-CAP1702I-E-K9 access points with a Cisco 2504 Wireless Controller.
The AIR-CAP1702I-E-K9 needed “IOS 8.0.100.0”. So we could not downgrade to 7.6.130.0.
I used this command on the controller: config ap dtls-wlc-mic sha1
After this command, the accesspoint were able to download the config and join the controller.
Great info Koos 🙂 will try in my lab.
I was banging my head against the wall for half a day, APs couldt’n join because of SHA2 issue, and I didn’t want to downgrade. “config ap dtls-wlc-mic sha1” fixed the problem. thanks a million
I was in the same situation as Keivan…..Koos, thanks a lot, you saved me more hours of troubelshooting!
Glad it helped 🙂
Hi guys. I´m facing a problem:
“AP has SHA2 MIC certificate – Using SHA2 MIC certificate for DTLS.”
My virtual controller has a version: 8.1x.
Anyone has a suggestion/ facing these problem?
Thanks a lot!
Hi,
Run this command on WLC “config ap dtls-wlc-mic sha1”.
If still not works then downgrade WLC software to 7.6.130.0.
Regards
RSCCIEW
Happened to me when the AP came pre-configured in bridge mode instead of local mode from the factory (had this with 1552’s and just recently 1532’s).
The way I fixed it (while running 8.120.0) is manually add the AP’s mac address (located on the sticker on the AP chassis) to the security/mac filtering area of the controller to force the controller to allow it to connect. It will still use SHA2.
Hi john,
Yes that’s normal behavior if a AP comes with a mesh image then u must need to enter MAC address of it to WLC and then change the AP mode.
Regards
Hi RSCCIEW , where could I download 7.6.130.0. ? it’s no longer available on cisco CCO download anymore Thanks !!!
I think it’s deferred from Cisco. You must try to download other version.
Regards
RSCCIEW
i had same issue and resolved by applying command on wlc as:
config ap dtls-wlc-mic sha2
Hi all,
I have CT5500 running 7.6.130 and AP1532 8.0.110
the Ap can’t join the WLC , i tried to do this command : config ap dtls-wlc-mic sha1″ but don’t take effect.
Hi,
I think you have a mesh image AP!!!
Did you add the MAC address of the AP in WLC? If not then add it and try again.
Regards
RSCCIEW
Hi,
i did not add MAC adress of the AP, but How to do that ?
hi,
yes i had mesh image AP so i manually add the AP’s mac address under security -> mac filtering area of the controller to force the controller to allow it to connect and now it works fine.
Thank’s a lot.
Best Regards
%DTLS-5-ALERT: Received WARNING : Close notify alert from 192.168.14.1
Create a thread on cisco support community and provide me the link .. So that we can find out the core issue.
Regards
RSCCIEW
Hi All,
Can you help me, I very frustated 😦
My AP can join to WLC, with error :
*Apr 2 04:35:40.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: x.x.x.x peer_port: 5246
*Apr 2 04:35:40.539: %CAPWAP-5-DTLSREQSUCC: DTLS connection created sucessfully peer_ip: x.x.x.x peer_port: 5246
*Apr 2 04:35:40.539: %CAPWAP-5-SENDJOIN: sending Join Request to x.x.x.x
Translating “CISCO-CAPWAP-CONTROLLER”…domain server (255.255.255.255)
*Apr 2 04:35:58.703: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to x.x.x.x:5246
*Apr 2 04:35:58.799: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to down
*Apr 2 04:35:58.811: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset
I try to used this command on the controller: config ap dtls-wlc-mic sha1, but not effect.
please help
Hi, Better to create a thread on CSC…. I will respond..
Thanks
I’m having the same issue. What was the resolution.
Please create a thread on CSC and paste the complete AP console logs.
Regards
RSCCIEW
Hi, very clear doc with clear explaination. But as per the ciso 7.6.130 is a deffered ios version. so will have any prob?
Dont go with any deffered version. Either use 8.0.152.0 or 8.5 version.
Regards
RSCCIEW