Migrate a vCenter Server and Change its IP Address

I needed to migrate a vCenter Server between datacenters. A new IP address was required for the vCenter Server at the destination. The process of changing a vCenter Server’s IP address became a straightforward process in vSphere 6.5. However, to add a complication in my situation, I also needed to migrate the vCenter Server. Unfortunately, the destination network was not available at the source. Here are the steps I went through and then an issue I ran into. This was for a 6.7 vCenter Server appliance with an embedded PSC.

  • Backup the vCenter Server
  • Shutdown vCenter Server
  • Clone vCenter Server
    • This was only a failsafe if the vCenter Server does not work at the destination to avoid restoring from a backup
  • Power on the vCenter Server
  • Run and save an export of RVTools
    • I always like to do this before big vCenter Server work so that I know where all my VMs are at
  • If only using a vDS, verify you have a port group with ephemeral binding
    • I did not need it, but you might depending on your destination
  • Change the IP address of the vCenter Server
    • The new IP address was displayed in the vCenter Server console
  • Shutdown the vCenter Server
  • Migrate the vCenter Server with VMware vCenter Converter Standalone to the destination ESXi host.
    • Ensure to verify and/or make the following changes in Converter
      • Required VM Hardware Version
      • VMXNET3
  • Change the DNS records for the vCenter Server
  • Power on the vCenter Server
  • Verify if the vCenter Server vNIC is connected
  • Reboot the vCenter Server
  • Verification
  • Delete the original and cloned vCenter Servers at the source

I thought everything went well. All ESXi hosts and VMs appeared to be happy. However, a user reported his remote console for some VMs would freeze after about 30-45 seconds and I also noticed some vDSs had sync issues. I did some research and found out about 90% of my hosts had the old vCenter Server IP address in vpxa.cfg.

VMware has KB1001493 which covers this issue. There are two methods to resolve the issue in the KB article. Every host with the issue needed to be touched so a lot of tedious work required for both methods. At first, I went with method 1 and tried it out on one host. The host was not responding after restarting the management agents and required to restart the vCenter Server service. I did not want to have all of my hosts not responding for a long period of time nor did I want to restart the vCenter Server service after restarting managements agents on each host. Therefore, I went with method 2.

Method 2 had more steps, but seemed to be cleaner. Essentially, each host is removed and added back to the vCenter Server one at a time so that seemed like a better approach. Hosts with a vDS required a little more work and documentation since a host will not have access to the vDS when removed. That meant to first put each host into maintenance mode before starting the first step. Then go through the steps and repeat on the next host. This greatly reduced the risk of VMs hitting any road bumps since VMs were not on a host that was being worked on. Then, of course, add each host back to the vDS. If no vDS and only standard switches, than no need to follow my extra step since network connectivity will be fine when the host is removed from the vCenter Server. Keep in mind performance data, permissions (depending what level they are set), VM placement in a folder, tags, and events/tasks (host level) are lost when removing a host from a vCenter Server. By the way, I did not do step 4 (Reinstall the VMware vCenter Server agent). The referenced KB article only mentioned up to 6.0, but seemed to worked well.

Perhaps, there’s a cleaner way to do this process. However, all things considered, the IP address change and migration went well. No outages and the desired outcome was achieved. I did two tasks at once so I, at first, thought that’s why I ran into the host issue. Though, I expect many people may have had this same issue, without combining the two tasks I did, since there is a KB article on how to resolve the issue.

VMware Training and VCPs/VCAPs Stats

VMware is giving away a free 6 month premium subscription to VMware Learning Zone for everyone. You need to register by November 6, 2020 and your subscription will expire 6 months after you register. That makes it easier to plan ahead to take advantage of a time frame that works best for you. You can even sign up with a personal email address. Check out the official article from VMware for more details and how to register.

VMware released details on the amount of VCPs and VCAPs. I like looking at statistics and it’s fascinating when vendors release numbers on certification holders. VMware last did this a few years ago. Unfortunately, I could not find the past numbers directly from VMware, but I did find it on vInfrastructure Blog. I will reference the data from the aforementioned sites in this article.

There are only 3,850 VCAPs in the United States and a total of 13,580 around the world. There are 98,576 VCPs in the United States and a total of 335,371 VCPs around the world. VCAPs have nearly tripled in the past few years and over 6 times more VCPs. I wonder if that is due to VMware changing their strict 2 year expiration on VCP certs.

The United States is on par with the world total of only roughly 4% of VCPs going ahead to achieve the VCAP certification. There are some overachieving countries like Egypt, which has 10% of VCPs getting their VCAP. Hopefully, VMware will give a list of totals in all countries. I made a comment requesting additional data and I will edit this article if I receive it.

VCAP-DCV Deploy 2020 Exam Experience

I passed the VMware Certified Advanced Professional 2018 – Data Center Virtualization Deployment exam (3V0-21.18) with a score of 400. I am glad the certification is considered 2020 so it doesn’t look as old already. I passed the VCAP-DCV Design exam a couple years ago. Oddly enough, the exam I just took earned me the VCIX badge for 6.5 and 2020. The Design exam also gave me the VCIX 6.0 badge a couple years ago when paired with my VCAP5. Funny how that worked out with the extra badge.

I received my results and badges the following day after I took the exam. I was pleasantly surprised to receive my results that fast, especially considering VMware says it can take 4-6 weeks. I received emails from VMware and Acclaim. I read some exam takers who didn’t receive an email from VMware and never even found out their score. They simply knew they passed only because they received the badge.

The exam begins as soon as you hit start and loads. You see what looks nearly identical to a VMware HOL lab and you can start reading the question manual before everything finishes loading. Then you have 210 minutes to complete everything, which is actually slightly longer than what the exam blueprint says. By the way, VMware says you cannot press the control key and right-click. Both worked for me.

I made my first run through of all 17 questions with about one hour left. I was only stumped on one question and a part of another. When I first took the VCAP exam four years ago, I didn’t have enough time and didn’t answer all the questions. Though, I still passed. I am not sure if the exam got easier or just my added experience got me through it much quicker. Anyway, I spent a half hour double checking my work and then finally clicked what to do with the one question I was stumped on. I was only stuck on one part of a question. I was fairly positive I did very well when I ended the exam. I don’t mean to sound braggadocious, but I honestly thought I would have had scored more than 400.

I might have lost some points on typos. The question manual was visually clear, but everything else was a little fuzzy within the environment. This made a lot of the text difficult to read and I couldn’t figure out a way to make the text sharper. Another problem was almost every time I hit a key, multiple letters were outputted. It was very annoying. I tried to drag and drop from the manual as much as possible. Other than that, the exam ran well. Much better than the past version of it.

I am sure most people practicing for the exam are using one vCenter Server. That’s fine. It’s easy to next your way through a wizard. However, remember that the exam has multiple vCenter Servers so you have to be very careful you are always working on the vCenter Server you are required to for the question. I knew this coming into the exam and I still had to go back to correct a mistake I made.

Here are a couple links to practice VCAP questions to try out. I forgot to post them in my last article. They were helpful to prepare for the exam to get in the mindset of possible questions. The HOLs they refer to are no longer available, but you can pull up other related HOLs to run through most of the questions or use your own lab.
https://www.jortechnologies.com/vcap-dcv-deploy-hol-based-simulator-free http://virtualg.uk/vcap6-dcv-deploy-exam-simulator-free

The exam sounds more daunting than it actually is. Most people with a lot of VMware experience can pass it with some studying and focusing on features they have never used. Be ready to stay focused for a few hours and you will be fine. Good luck!

Studying for VCAP6-DCV Deploy Certification

I have been studying for the VMware Certified Advanced Professional 2018 – Data Center Virtualization Deployment exam (3V0-21.18) for the past couple of months. It is based on vSphere 6.5. I just found out the vSphere 6.7 version of the exam is now out (3V0-22.19). The 6.5 version retires on February 28th so I need to take the exam soon. I rather keep pushing forward than change the exam I will take since I am far along with studying. One good thing is passing either test counts towards VCAP-DCV Deploy 2020. There was never a VCAP-DCV 2019 so passing the exam last year would have still been called 2018. Makes the certification look old if passed last year when it should have not. Though, no need to worry now.

Of course, the best place to start studying is looking at VMware’s exam blueprint to get an overview of the exam. The 6.5 and 6.7 versions both have identical objectives on their blueprints. Perhaps nothing major has changed between the exams.

I recommend watching a presentation by Joshua Andrews. He discussed all of VMware’s certifications and focuses on the VCAPs. The video is from December 2018 so it’s still kind of recent. He use to work on making certification exams at VMware so I think he is a great source to learn from. He also has an excellent blog and an article on VCAP exam links.

Ricardo Conzatti has an awesome exam simulator. He gives you a legit environment to RDP to and lab questions to solve in fleshed out scenarios. Then he will give you the answers after your exam simulator session. First, you need to schedule and pay $10. He received a lot of no shows in the past so understandable he wants to try to hold people accountable. Make sure to check your email’s spam folder. I thought I didn’t receive an email for how to connect the day of my scheduled session, but turned out the email was in my spam folder and I lost a little time to eventually find it.

VMware HOL is also a great place to try out your skills. Since the exam environment is based on the HOL, you can get a good feel with how the interface will look and do some tweaks to get the screen to look good. The challenge and vSAN related labs would be good to do. See how it is to have HOL open on one screen and to go back and forth with the manual.

I will be sure to post when I have the results of my exam. I will share my experience and anything else I come across that may be helpful.

VMworld 2019 – The Tourist

VMworld 2020 will be returning to San Francisco next year. Back at the Moscone Center starting August 31st through September 3rd. I figured it would still be valid to post a touristy article. Perhaps, some others will be interested in doing activities outside of VMworld. I stayed in San Francisco until Sunday and had a lot of fun.

Thursday was the last day of VMworld. The solutions exchange and many other areas were closed this day. However, there was still a good half day of sessions and such. I didn’t waste any time after VMworld to start sightseeing that same day.

I started off by going to Pier 39. There were many attractions to check out; sea lions, little stores, and restaurants. The main reason I went there was to go to the Aquarium of the Bay. It was nice, but I feel there are better options if you are short on time. The best part was the tunnel with animals surrounding me from the top and sides. The aquarium ticket came with a free ticket to Madame Tussauds, which was nearby.

Friday was one of only full two days for sightseeing so I wanted to maximize what I could do. I started off with the San Francisco Zoo. It was well worth the drive from downtown. The zoo was even less expensive than the aquarium and spent twice as much time at the zoo. The zoo had a very nice African exhibit among many others. The Pacific Ocean was just a few minute walk so I definitely had to check it out being that I am from the east coast.

The Golden Gate Park was up next for Friday afternoon. I wish I had more time and energy to spend there. It’s 3.5 miles wide so a lot of ground to cover. The California Academy of Sciences is within the park and was my favorite attraction. It had everything there; a rain forest, an aquarium, a planetarium, an earthquake simulator, and a natural history museum. I recommend having at least three hours there and to pick up the included planetarium ticket when you first arrive. That way you can choose what time and specific show you want to see. The rain forest and aquarium were absolutely beautiful.

Then I ended the night for dinner in Chinatown. Amazing that it’s the biggest Chinatown in the United States. Fun to walk around there. A lot of interesting foods and stores to check out.

I wanted to cover a lot of ground on Saturday so I rented a car. I first made a quick stop at the painted ladies, which is the iconic houses displayed in the opening credits for Full House. Then I went to Coit Tower for an amazing view of the city. Be prepared to wait for a little while to get to the top of the tower and not much parking on the top of the hill. Lombard Street, the famous winding street, is nearby so a good opportunity to drive down the road. Finally finished this area with lunch at Ghirardelli Square.

A lot of people recommended going to Muir Woods National Monument. It is the closest area to see redwoods. I went over the Golden Bridge to get there and I wasn’t impressed with the bridge. It seemed like a lot of other brdiges to me. Muir Woods was very nice. Amazing to see the huge trees. After going past the visitor center, there were a few looped trails. Boardwalks paved the way for some of it so it was an easy walk. The best advice I can give is to make a parking reservation as far in advance as you can. The reservation is only $8 and there are not many other parking options.

The last stop of the day was at Twin Peaks. It’s the second highest mountain in the city. I call it the lazy man’s hike because there’s a road that drives towards the top. Then there are several places to park and can hike to the summit. A nice view from a different side of the city.

I tried to make the most of my trip from the work side to the personal side. I greatly enjoyed my trip. I recommend everyone to venture deeper into the city next VMworld.

VMworld 2019 – The Fun and Random Side

VMworld is much more than the business side. I wanted to take moment to go over somethings outside of the usual keynotes, sessions, labs, etc. Also, to mention some other information and tips that didn’t fit in my other articles.

I recommend getting to San Francisco on Saturday to get the most out of the event. Give yourself some time to get settled in. There were a lot of events on Sunday, especially if you’re a TAM customer. Then you can be one of the first to hit the solutions exchange that night. Depending on your schedule, you may not have a lot of time the rest of the week for the solutions exchange. Also, more likely you will get a t-shirt in your size 🙂

Registration was quick and easy. I never saw a long line there whenever I passed by. Don’t go out of the way to pick up the official bag until you need to go to the west building. There were plenty of them so save the steps.

Be prepared to walk a lot. I was walking around 15,000 to 20,000+ steps a day. Even more tiring when carrying a bag of swag or a laptop all day. Wear sneakers and sit down whenever you can.

I never saw a session full. I don’t even think I saw a room 80% full. This is excluding something like a session with a very limited amount of seats, such as a raspberry pi based session. I am not a fan of how session reservations work. Don’t let a session marked as full to hold you back. Just have a backup plan with something else to do if it is actually full. Also, you can always watch the session online later on.

The breakfast and lunches provided by VMware were not good and lacked variety. Greasy breakfast sandwiches and pastries in the mornings. Mostly sandwiches for four days in a row for meat-eaters. The vegetarian and gluten-free options were combined for some odd reason. That one was some sort of tofu salad from what I saw on some days.  The lunches had bulky one-use plastic containers that were a huge waste. I hope VMware/Moscone Center can have a more environmentally friendly version in the future. There are a bunch of restaurants nearby that are better alternatives.

There is no shortage of entertainment. Multiple parties every night. Then the big VMworld Fest on Wednesday night. Billy Idol and One Republic were great. I don’t think the performers knew what the concert was for. Hopefully, VMware can book a nicer venue next time.

I did not worry once about my safety outside of the Moscone Center or anywhere in the city. Several people warned me about aggressive homeless people and issues on the streets. I did not see anything different from where I live. I am guessing people that were worried probably live in a rural area. Obviously, take normal safety precautions. I definitely recommend to branch out of the area and not be worried. My final article next week will be on fun things to do around the city.

VMworld 2019 – The Business Side
VMworld 2019 – Solutions Exchange
VMworld 2019 – The Fun and Random Side

VMworld 2019 – Solutions Exchange

I was amazed by the size of the solutions exchange, but surprised with the limited hours it was opened throughout the week. I wish I would have had more time to talk to additional companies. Fascinating to see what companies had to offer and what gimmick to get people to listen. A few companies had magicians. A lot of companies had a raffle at the end of a 10-15 minute presentation, which reoccurred often.

I thought it was amusing the lengths backup companies went with their booths. Rubrik and Cohesity both had two-story booths and sport related contests. Commvault had a therapy dog park between the Moscone buildings.

I wanted to share some photos of the most interesting areas of the solution exchange and also the square.

VMworld 2019 – The Business Side
VMworld 2019 – Solutions Exchange
VMworld 2019 – The Fun and Random Side

VMworld 2019 – The Business Side

VMworld2019_GS1

I went to VMworld for my first time and had a blast. The event was great, but not without its flaws. I tweeted during the event and I may start tweeting more often. Also, when I post a new article. However, I never got around to posting an article during the event.

Monday’s general session presentation fell flat for me. Project Pacific and Tanzu Mission Control were already announced earlier that day. Then the keynote felt too scripted. After looking past the superficial side, the content was huge. Project Pacific will have Kubernetes embedded in the hypervisor. This is a great move by VMware to work more closely where the industry has been heading. Their latest acquisitions definitely tell what they are up to; Pivotal and Carbon Black. Then Tanzu Mission Control will give central management for Kubernetes clusters on-prem and in the cloud.

Tuesday’s general session highlighted some of the prior day’s announcements and went into additional news. VMware Cloud on Dell EMC is now available. It was previously announced at Dell’s conference, but this is the first time I heard about it. Dell will send an engineer to deploy a rack of hardware at a customer’s location and it shows up in a customer’s SDDC. Then Dell manages the hardware and ESXi stack. I really like this model. The best both worlds in my mind; a cloud-like architecture on-prem. NSX Intelligence was announced. VMware’s new CTO, Greg Lavender, was even announced on stage.

There was no shortage of sessions. I liked that VMware had each session labeled according to the technical level. My company has a TAM so I was able to attend additional sessions, which were under an NDA. I attended mostly them since they are not recorded. I am looking forward to watching many of the recorded sessions later on. Below are my favorite non-TAM sessions that I attended.

William Lam and Emad Younis hosted a great session on ‘The Next Generation of Lifecycle Management for vCenter Server’. Here are a couple nice additions to vSphere 6.7, which are currently available. Display the topology view of all vCenters and PSCs in an SSO domain (U2). This is under administration and system configuration. Now able to change the hostname and IP address of a vCenter Center (U3). Then they went into potential future additions in tech preview. A better summary page for vCenter, which includes; notification for vCenter updates, last updated, last backed up by the native file-based backup, summary totals at this level, and overall health status from the 5480 page. The screenshot below is from their presentation. They said there will be a permission to set to enable the update available notification so it can be easily hidden from a client or whoever. My favorite tech preview was the vCenter interoperability built into vCenter. No more going to the HCL for some components. Some potential new features are desired state for vCenter, improved host profiles, and easier NSX install. I am looking forward to all of this. Hopefully in the not too distant future.

vCenterTechPreview.PNG

I attended the Intro to Raspberry Pi and Run K8s on VMware. I figured both would be good since they were hands-on. I already have a Raspberry Pi running RetroPi. I figured this would help me get more motivated to do something else with my new Raspberry Pi I received in my vExpert bag. I definitely want to think of a home project for it. The Kubernetes session had a lab and was actually the first time I got to use Kubernetes. It served as a good intro. The lab was on VMC on AWS so it was legit.

I decided I will work a few more articles on VMworld. One of the solution exchange. Then the fun side of VMworld. The final will be on personal travel after VMworld. I stayed a few extra days to check out the city. I thought this may help others interested in doing the same since VMworld 2020 is back August 31st to September 3rd at the Moscone Center in San Francisco.

VMworld 2019 – The Business Side
VMworld 2019 – Solutions Exchange
VMworld 2019 – The Fun and Random Side