Is it possible to update only the phpvms files that have been modified, without having to delete the vendor file?
You would be best served to follow the instructions in the docs to the letter.
It’s quick and easy once you do it a few times.
Why don’t you want to delete the vendor folder?
Not advised (or accepted) but possible as long as you can keep track of every commit and/or every changed file.
In the end, what is published is the best and approved way to update, it ensures smooth installation of latest build and eliminates human errors. If you want to do it the hard way around, then so be it But be advised, if something goes wrong in your process and you start getting strange results and/or unexpected errors, this method will reduce the support you will get back
Good luck
PS: If it is only the vendor folder you are worrying about, it is easy to build it with composer commands and with proper entries in composer files if you are using additional packages on your server. You can even build vendors on a local server and then upload to your real server if it does not offer command line access etc.