Correct and/or improve handling of certain quiesced snapshot failures

Bug #1814832 reported by Oliver Kurth
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
open-vm-tools (Debian)
Fix Released
Unknown
open-vm-tools (Ubuntu)
Fix Released
High
Unassigned
Bionic
Fix Released
Undecided
Oliver Kurth
Cosmic
Fix Released
Undecided
Oliver Kurth

Bug Description

[Impact]

 * Upstream identified an issue that can occur on aborted (or
   due to communication issues while doing) quiesced snapshots.

 * Backport the upstream changes as part of our work getting the latest
   10.3.5 to the latest Ubuntu LTS (Bionic)

[Test Case]

 * This is hard to test, but fortunately VMWare who have the right setup
   for this tested our change from a PPA. I'll ask for that again on SRU.
   Never the less I'll outline roughly what is needed to trigger [1]:
   1. Use the host side interface to trigger a quiesced snapshot
   2. this is the hard part - have communication failures between vmtools
      (guest) and VMX (host) while this is ongoing.
   3. From the Hosts POV the operation is aborted, but vmtools sends a
      manifest eventually
   4. Receiving this will make VMX reply a error (as it didn't wait for
      anything like it)
   5. Finally this broke the state machine and in subsequent cases vmtools
      will not send a manifest again
 * Further related fixes make sure vmtoolsd give up if VMX aborted the
   snapshot [2] and another [3] makes sure manifests are always sent to
   avoid any desync between VMX and vmtoolsd

[1]: https://github.com/vmware/open-vm-tools/commit/a1306fcbb6de6eae5344d5d74747068ea89aa5fc
[2]: https://github.com/vmware/open-vm-tools/commit/0c9174716ba828899418ba07efc3aab0bff004cc
[3]: https://github.com/vmware/open-vm-tools/commit/c31710b3942f48b1c11ebde36f34e7e159d1cbf0

[Regression Potential]

 * This is quite a change to the snapshot handling, so in theory there a
   regression has to be assumed. Due to a lack of testcases and expertise
   on our side that was handed to VMWare itself who have a much wider
   matrix of tests and setups to run them on.
   This was tested and confirmed good (even before the change made
   it upstream).
 * Furthermore those kind of snapshots are relevant to those
   who use them (and they most likely want the fix for reliability as you
   could get into a state where no further snapshots were possible). But
   OTOH the majority of users of the open-vm-tools package most likely
   don't use the feature at all. Fortunately changes are local to only the
   vmbackup functionality.

[Other Info]

 * n/a

---

Customers may hit issues with quiesced snapshots under certain circumstances. This is fixed in a branch forked from 10.3.5:

https://github.com/vmware/open-vm-tools/tree/stable-10.3.5-quiesced-snapshot

A more detailed description of the issue can be found in the individual commit messages.

Also filed at Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=921470

Related branches

Revision history for this message
Oliver Kurth (okurth-1) wrote :

A short summary of the changes:

- Attempt to notify the host that a backup manifest is available on every completed snapshot. Previously vmtoolsd attempted to detect whether the host was running an older version of ESX that did not support receiving a backup manifest, which occasionally resulted in it erroneously identifying a host as an older host.
- Provide more informative logging messages when a backup manifest notification is refused by the host.
- Abort quiesced snapshot operations promptly after detecting that the host has aborted its side of the operation. Previously vmtools would continue until it hit its own internal timeout.
- Don’t try to notify the host that a backup manifest is available when the quiesced snapshot is aborted.

We suggest to have these changes integrated into the 10.3.5 package.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi Oliver,
thanks for the report and the Debian bug.
We can wait a few days if Bernd gets to integrate that in Debian to get it in Disco/19.04.
Otherwise I might cherry pick this before Feature Freeze (this is isn't a feature so it could be later, but uploads are easier before the freeze).

Since we are still in "preparation" of the 10.3.5 backports themselves in bug 1813944 and I heard that your verification will take a while due to chinese new year that actually fits quite well.

I'd want to do the following:
- wait a week what Bernd says for Debian which would get into Disco
- if not actioned then consider a Delta for Disco
- wrap that up into the PPA for the Bionic/Cosmic SRUs before verification

Most likely we would rely on you to test this bug here in regard to all the VMWare Hipervisor involvement that is needed for the qiuesced snapshots. But for the SRU processing it would be nice if you'd describe as good as possible how such a test would look like. That description you could add right now so that I can pre-prepare the SRU template already.

Changed in open-vm-tools (Ubuntu):
importance: Undecided → High
status: New → Triaged
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi Oliver,
Bernd was rather open on debian #921470 but you'd need to reply if there is a branch with this (and more) stable things or if this is the only one reasonably backpporting to 10.3.5 for now.

I think once you have answered that he will upload and we will sync to Disco (and consider SURs from there). So it would be really helpful to resolve that open question.
Since the bug here is blocked on that as well I'm marking it incomplete for now.

Changed in open-vm-tools (Ubuntu):
status: Triaged → Incomplete
Changed in open-vm-tools (Debian):
status: Unknown → New
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Version 2:10.3.5-6 is in disco-release so that is done.
Since that worked as expected I can also bundle it into the planned SRU to backport to 18.04/18.10.

Changed in open-vm-tools (Ubuntu Cosmic):
status: New → Fix Released
Changed in open-vm-tools (Ubuntu Bionic):
status: New → Fix Released
Changed in open-vm-tools (Ubuntu Cosmic):
status: Fix Released → Triaged
Changed in open-vm-tools (Ubuntu Bionic):
status: Fix Released → Triaged
Changed in open-vm-tools (Ubuntu):
status: Incomplete → Fix Released
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

The backports for Bionic and Cosmic are prepared and bundled with bug 1813944 as planned.
As usual please pre-verify this against the PPA [1] before we upload this to the SRU queue.

Setting incomplete as we wait for feedback on this.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3617

Changed in open-vm-tools (Ubuntu Bionic):
status: Triaged → Incomplete
Changed in open-vm-tools (Ubuntu Cosmic):
status: Triaged → Incomplete
Changed in open-vm-tools (Debian):
status: New → Fix Released
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Checks complete (in other bug)

Uploaded to Bionic/Cosmic SRU queue now.
Once the SRU Team accepts it they will call for the next (last) round of testing.

Changed in open-vm-tools (Ubuntu Bionic):
status: Incomplete → In Progress
Changed in open-vm-tools (Ubuntu Cosmic):
status: Incomplete → In Progress
Revision history for this message
Steve Langasek (vorlon) wrote : Proposed package upload rejected

An upload of open-vm-tools to cosmic-proposed has been rejected from the upload queue for the following reason: "needs a separate SRU bug for the debian/desktop.conf addition. Also, upload fails to comply with https://wiki.ubuntu.com/DebianMaintainerField".

Revision history for this message
Steve Langasek (vorlon) wrote :

An upload of open-vm-tools to bionic-proposed has been rejected from the upload queue for the following reason: "needs a separate SRU bug for the debian/desktop.conf addition. Also, upload fails to comply with https://wiki.ubuntu.com/DebianMaintainerField".

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Temporarily blocked to clean up some issues on the SRU - thanks Steve for letting me know!

Changed in open-vm-tools (Ubuntu Bionic):
status: In Progress → Triaged
Changed in open-vm-tools (Ubuntu Cosmic):
status: In Progress → Triaged
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Added an SRU Template here as well, while we did drive this more as an MRE in the past while I was cleaning up on Steve's request lets make all related bugs proper SRUs to be sure on the next round.

description: updated
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

After improving on what we have found triggered by the first SRU review this is ready again.
Changes:
- Despite being an MRE in general all associated individual bugs have full SRU templates now.
- maintainers correctly updated
- The change to the vgauth Dependencies is now safer

That said, this is uploaded to Bionic/Cosmic unapproved again and waiting for SRU Team re-review.

Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Oliver, or anyone else affected,

Accepted open-vm-tools into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-vm-tools/2:10.3.5-7~ubuntu0.18.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in open-vm-tools (Ubuntu Cosmic):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-cosmic
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Oliver, or anyone else affected,

Accepted open-vm-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-vm-tools/2:10.3.5-7~ubuntu0.18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in open-vm-tools (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thanks Steve for accepting.

@Oliver - verifying the Snapshot case on Bionic and Cosmic is up to you since you have the setup to do so.

Changed in open-vm-tools (Ubuntu Bionic):
assignee: nobody → Oliver Kurth (okurth-1)
Changed in open-vm-tools (Ubuntu Cosmic):
assignee: nobody → Oliver Kurth (okurth-1)
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

@Oliver - any update on this testing?
That is the only one missing still to release this.

Revision history for this message
Oliver Kurth (okurth-1) wrote : Re: [Bug 1814832] Re: Correct and/or improve handling of certain quiesced snapshot failures
Download full text (5.9 KiB)

I think this is good to go, and does not need any more additional testing, assuming you have applied the changes from the stable-10.3.5-quiesced-snapshot branch . All these changes are also in the 10.3.10 release, which has gone through our testing already. I also did test your packages before with the same changes.

Thanks,
Oliver

________________________________
From: <email address hidden> <email address hidden> on behalf of Christian Ehrhardt  <email address hidden>
Sent: Tuesday, April 9, 2019 11:10 PM
To: Oliver Kurth
Subject: [Bug 1814832] Re: Correct and/or improve handling of certain quiesced snapshot failures

@Oliver - any update on this testing?
That is the only one missing still to release this.

--
You received this bug notification because you are subscribed to the bug
report.
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.launchpad.net%2Fbugs%2F1814832&amp;data=02%7C01%7Cokurth%40vmware.com%7C4609d9c88c5845489b4408d6bd7cae0e%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636904740527856210&amp;sdata=mH6Si7A4rok4WelYvcvT42Y5nQY85huMEf8udAfGsSY%3D&amp;reserved=0

Title:
  Correct and/or improve handling of certain quiesced snapshot failures

Status in open-vm-tools package in Ubuntu:
  Fix Released
Status in open-vm-tools source package in Bionic:
  Fix Committed
Status in open-vm-tools source package in Cosmic:
  Fix Committed
Status in open-vm-tools package in Debian:
  Fix Released

Bug description:
  [Impact]

   * Upstream identified an issue that can occur on aborted (or
     due to communication issues while doing) quiesced snapshots.

   * Backport the upstream changes as part of our work getting the latest
     10.3.5 to the latest Ubuntu LTS (Bionic)

  [Test Case]

   * This is hard to test, but fortunately VMWare who have the right setup
     for this tested our change from a PPA. I'll ask for that again on SRU.
     Never the less I'll outline roughly what is needed to trigger [1]:
     1. Use the host side interface to trigger a quiesced snapshot
     2. this is the hard part - have communication failures between vmtools
        (guest) and VMX (host) while this is ongoing.
     3. From the Hosts POV the operation is aborted, but vmtools sends a
        manifest eventually
     4. Receiving this will make VMX reply a error (as it didn't wait for
        anything like it)
     5. Finally this broke the state machine and in subsequent cases vmtools
        will not send a manifest again
   * Further related fixes make sure vmtoolsd give up if VMX aborted the
     snapshot [2] and another [3] makes sure manifests are always sent to
     avoid any desync between VMX and vmtoolsd

  [1]: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fvmware%2Fopen-vm-tools%2Fcommit%2Fa1306fcbb6de6eae5344d5d74747068ea89aa5fc&amp;data=02%7C01%7Cokurth%40vmware.com%7C4609d9c88c5845489b4408d6bd7cae0e%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636904740527856210&amp;sdata=jfveRSloyf1AgyBJxg8jmLLGo3RJv7e%2F3%2FTbkEHPcTI%3D&amp;reserved=0
  [2]: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2...

Read more...

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hey Oliver,
TL;DR: sorry, but please test it from proposed (again) in Bionic&Cosmic.

Detail:
Unfortunately this is not how it works.
I'd want to mark it verified per your request, but I know that this will raise SRU-Team-Eyebrows and not be released.

I'm in fact already in discussion for a couple of cases where people pre-tested on PPAs and we later released as SRU which means they need to be verified again for the final releasing. The problem is the tradeoff on non-trivial bugs, you want:
 - provide something to test for the reporter ASAP
 - be able to iterate on alternatives
 - run more tests and tests by different parties
 - all of the above you want to do without yet polluting -proposed (as there other
   builds could depend on it, it could break other proposed testing, ...)
 => This is what a PPA is used for most of the times.

But due to the above that usually means we end up with a change tested and verified from a PPA that will then in this way be proposed as SRU (same code + some paperwork).

There the process requires that this final build shall be verified (yes - again).

This mostly comes from the past were people didn't pre-check with PPAs, but also is needed to cover any gap as the testing of the PPAs might have been:
- weeks ago
- ran with other dependencies at the time but would fail now
- built with other dependencies at the time
- only one release was tested but the SRU covers multiple
But for the above and more it usually is worth and by the process required to re-test the versions in proposed.

I've had a few such cases in the past now and I have started discussing alternatives, which so far all come down to do the tradeoffs above at a different place - or to have the SRU process accept verifications with identical builds (as in this case). But there seems to be no easy best answer. Resolving that is a long term effort ending in adaption of the SRU or Developers process - nothing that is done in a minute.

Until all of that is resolved, I'd really ask you to re-test the case as built in Bionic/Cosmic-proposed and I hope that is not too much of a burden/effort for you.

Revision history for this message
Oliver Kurth (okurth-1) wrote :
Download full text (8.0 KiB)

Hi Christian,

is this the the repo for the packages: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3617

I am a little confused since these are from Feb 12th, and I believe I had them tested already (but can't find the email that I sent). So I just want to make sure I am not testing the wrong ones. Also, where can I find the packages for cosmic?

Thanks,
Oliver

________________________________
From: <email address hidden> <email address hidden> on behalf of Christian Ehrhardt  <email address hidden>
Sent: Wednesday, April 10, 2019 11:50 PM
To: Oliver Kurth
Subject: [Bug 1814832] Re: Correct and/or improve handling of certain quiesced snapshot failures

Hey Oliver,
TL;DR: sorry, but please test it from proposed (again) in Bionic&Cosmic.

Detail:
Unfortunately this is not how it works.
I'd want to mark it verified per your request, but I know that this will raise SRU-Team-Eyebrows and not be released.

I'm in fact already in discussion for a couple of cases where people pre-tested on PPAs and we later released as SRU which means they need to be verified again for the final releasing. The problem is the tradeoff on non-trivial bugs, you want:
 - provide something to test for the reporter ASAP
 - be able to iterate on alternatives
 - run more tests and tests by different parties
 - all of the above you want to do without yet polluting -proposed (as there other
   builds could depend on it, it could break other proposed testing, ...)
 => This is what a PPA is used for most of the times.

But due to the above that usually means we end up with a change tested
and verified from a PPA that will then in this way be proposed as SRU
(same code + some paperwork).

There the process requires that this final build shall be verified (yes
- again).

This mostly comes from the past were people didn't pre-check with PPAs, but also is needed to cover any gap as the testing of the PPAs might have been:
- weeks ago
- ran with other dependencies at the time but would fail now
- built with other dependencies at the time
- only one release was tested but the SRU covers multiple
But for the above and more it usually is worth and by the process required to re-test the versions in proposed.

I've had a few such cases in the past now and I have started discussing
alternatives, which so far all come down to do the tradeoffs above at a
different place - or to have the SRU process accept verifications with
identical builds (as in this case). But there seems to be no easy best
answer. Resolving that is a long term effort ending in adaption of the
SRU or Developers process - nothing that is done in a minute.

Until all of that is resolved, I'd really ask you to re-test the case as
built in Bionic/Cosmic-proposed and I hope that is not too much of a
burden/effort for you.

--
You received this bug notification because you are subscribed to the bug
report.
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.launchpad.net%2Fbugs%2F1814832&amp;data=02%7C01%7Cokurth%40vmware.com%7C2214f621df154380388a08d6be4c278c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636905631628753788&amp;...

Read more...

Revision history for this message
Oliver Kurth (okurth-1) wrote :

Oh, nevermind. Installing from -proposed.

Revision history for this message
Oliver Kurth (okurth-1) wrote :

Okay, I tested the quiesced snapshots, both for bionic and cosmic, using open-vm-tools 2:10.3.5-7~ubuntu0.18.04.1 and 2:10.3.5-7~ubuntu0.18.10.1. I looked at debug logs, everything is normal.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Yes testing -proposed was the right thing.
Thanks Oliver for your support, marking it verified then.

tags: added: verification-done verification-done-bionic verification-done-cosmic
removed: verification-needed verification-needed-bionic verification-needed-cosmic
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (5.1 KiB)

This bug was fixed in the package open-vm-tools - 2:10.3.5-7~ubuntu0.18.10.1

---------------
open-vm-tools (2:10.3.5-7~ubuntu0.18.10.1) cosmic; urgency=medium

  * Backport recent open-vm-tools (LP: #1813944)
    - also adresses handling of quiesced snapshot failures (LP: #1814832)
    - also adresses issues with resolutionKMS plugins sometimes fails to
      load at boot (LP: #1818473)

open-vm-tools (2:10.3.5-7) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [71b468f] make vgauth service execution more reliable.
    Since d3d47039 "Start vgauth before vmtoolsd" there is a potential race
    of starting vgauth so early that it might have issues. This was
    discussed back in the day in [1] to [2], but confirmed to be ok by
    VMWare.
    We were all somewhat convinced by this, but a bad feeling remained not
    only with me but also with Bernd [4].
    A recent SRU review denial made me rethink all of it and I think we can
    make it safer without thwarting the purpose of the original change.
    Note: Disambiguation of service names used below:
    vgauth - open-vm-tools.vgauth.service
    vmtoolsd - open-vm-tools.service
    fs - systemd-remount-fs.service
    tmp - systemd-tmpfiles-setup.service
    cloud-init - cloud-init-local.service
    Currently we have these dependency requirements:
    - vgauth should be before vmtoolsd
    - cloud init should be before vmtoolsd
    - cloud init has to be really early in general
      - therefore this is using DefaultDependencies=No
    That lead to this graph:
     fs / tmp -> vmtoolsd -> cloud-init
    And d3d47039 added it to be like:
     fs / tmp -> vmtoolsd -> cloud-init
                   ^
     vgauth --|
    But there is no need to have vgauth without any pre-dependencies at all.
    It is only needed to be "before" vmtoolsd, therefore we can make it:
     fs / tmp -> vgauth -> vmtoolsd -> cloud-init
    That will make execution of vgauth much less error-prone (even though I
    have no hard issue to report) while at the same time holding up all
    known required ordering constraints.
    [1]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/3
    [2]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/12
    [3]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/25
    [4]: https://github.com/bzed/pkg-open-vm-tools/pull/15#issuecomment-447237910
    Signed-off-by: Christian Ehrhardt <email address hidden>

open-vm-tools (2:10.3.5-6) unstable; urgency=medium

  * [43ec618] Correct and/or improve handling of certain quiesced
    snapshot failures.
    Thanks to Oliver Kurth (Closes: #921470)

open-vm-tools (2:10.3.5-5) unstable; urgency=medium

  * [54cce3e] Start vmtoolsd after apparmor.service.
    Github issue #17

open-vm-tools (2:10.3.5-4) unstable; urgency=medium

  [ Alf Gaida ]
  * [e13792d] udevadm trigger should not fail (Closes: #917642)

open-vm-tools (2:10.3.5-3) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [d3d4703] Start vgauth before vmtoolsd.
    VGAuthService needs to be ready when vmtoolsd runs. Certain cases - e.g.
    Site Reco...

Read more...

Changed in open-vm-tools (Ubuntu Cosmic):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for open-vm-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (5.1 KiB)

This bug was fixed in the package open-vm-tools - 2:10.3.5-7~ubuntu0.18.04.1

---------------
open-vm-tools (2:10.3.5-7~ubuntu0.18.04.1) bionic; urgency=medium

  * Backport recent open-vm-tools (LP: #1813944)
    - also adresses handling of quiesced snapshot failures (LP: #1814832)
    - also adresses issues with resolutionKMS plugins sometimes fails to
      load at boot (LP: #1818473)

open-vm-tools (2:10.3.5-7) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [71b468f] make vgauth service execution more reliable.
    Since d3d47039 "Start vgauth before vmtoolsd" there is a potential race
    of starting vgauth so early that it might have issues. This was
    discussed back in the day in [1] to [2], but confirmed to be ok by
    VMWare.
    We were all somewhat convinced by this, but a bad feeling remained not
    only with me but also with Bernd [4].
    A recent SRU review denial made me rethink all of it and I think we can
    make it safer without thwarting the purpose of the original change.
    Note: Disambiguation of service names used below:
    vgauth - open-vm-tools.vgauth.service
    vmtoolsd - open-vm-tools.service
    fs - systemd-remount-fs.service
    tmp - systemd-tmpfiles-setup.service
    cloud-init - cloud-init-local.service
    Currently we have these dependency requirements:
    - vgauth should be before vmtoolsd
    - cloud init should be before vmtoolsd
    - cloud init has to be really early in general
      - therefore this is using DefaultDependencies=No
    That lead to this graph:
     fs / tmp -> vmtoolsd -> cloud-init
    And d3d47039 added it to be like:
     fs / tmp -> vmtoolsd -> cloud-init
                   ^
     vgauth --|
    But there is no need to have vgauth without any pre-dependencies at all.
    It is only needed to be "before" vmtoolsd, therefore we can make it:
     fs / tmp -> vgauth -> vmtoolsd -> cloud-init
    That will make execution of vgauth much less error-prone (even though I
    have no hard issue to report) while at the same time holding up all
    known required ordering constraints.
    [1]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/3
    [2]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/12
    [3]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/25
    [4]: https://github.com/bzed/pkg-open-vm-tools/pull/15#issuecomment-447237910
    Signed-off-by: Christian Ehrhardt <email address hidden>

open-vm-tools (2:10.3.5-6) unstable; urgency=medium

  * [43ec618] Correct and/or improve handling of certain quiesced
    snapshot failures.
    Thanks to Oliver Kurth (Closes: #921470)

open-vm-tools (2:10.3.5-5) unstable; urgency=medium

  * [54cce3e] Start vmtoolsd after apparmor.service.
    Github issue #17

open-vm-tools (2:10.3.5-4) unstable; urgency=medium

  [ Alf Gaida ]
  * [e13792d] udevadm trigger should not fail (Closes: #917642)

open-vm-tools (2:10.3.5-3) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [d3d4703] Start vgauth before vmtoolsd.
    VGAuthService needs to be ready when vmtoolsd runs. Certain cases - e.g.
    Site Reco...

Read more...

Changed in open-vm-tools (Ubuntu Bionic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.