I've taught several upper-level courses recently, and have considered giving take-home exams to reduce the time pressure in the classroom.
However, their homework already consists of proving various exercises similar to what I would imagine to be on a takehome exam.
How is a takehome exam different from regular homework?
A homework assignment is there to help the students learn. A takehome exam is a test of the students' abilities and understanding.
From a practical standpoint, a few of the main differences are:
I usually encourage students to work together in groups on solving homework problems. By contrast, students must work individually when solving problems on a takehome exam.
When a student comes to office hours with questions about a homework problem, I will usually give them a hint every time they get stuck, working with the student until they are finally able to solve it. By contrast, I will usually not give any significant hints for problems on a takehome exam, except for very basic things like "you might find some of the theorems in chapter 3 helpful", or "there was a homework problem kind of like this a few weeks ago".
Similarly, I am usually willing to critique rough drafts of written proofs for homework problems during office hours. I will not critique any writing for a takehome exam until I grade it.
I usually make the problems on a takehome exam a little easier than homework problems, and the takehome exam counts more than a weekly homework.
Philosophically speaking, I tend to think of the students' homework grades as measuring the amount of effort they put into the course. A student who works with other students, comes to all of my office hours, shows me rough drafts of their proofs and so forth really ought to be able to solve all of the homework problems and write them up well. By contrast, a takehome exam really tests the student's understanding of the material, general problem-solving ability, and ability to write proofs clearly without help.
I'm not too excited about take-home exams, precisely for the reasons mentioned by OP. I prefer to have a clear-cut difference between homework (show that you can do the computations, write down clean and documented proofs, ...) and exams (show that you understand the subject matter, are able to explain it and note relationships, say what is applicable to a given situation, ...). If the matter turns out just too long, what I have done is to split one (or even various) problems up. I.e., do step one starting from this; assume that at the beginning of an iteration you have ... and show the state at the end of the iteration; if the state is this one, is the iteration done (and what is the result). You don't want to check if they can do all the calculations in detail, you are interested in assessing their understanding.
Yes, time pressure is a problem. I schedule exams for saturdays (rooms are empty,it is easy to get enough so the "look a the neighbor's work" isn't too much of a problem, and it is also possible to reserve rooms for two lecture blocks in a row (to have leeway for getting everyone seated, exams handed out, any last minute instructions, possible stragglers, ...). This reduces time pressure. If the estimate of the time turns out too short, there is no risk of getting thrown out by the next class.
