When should I use design based on photos and when based on illustrations? Is it based on the emotional impact which we designers want to achieve?
The main considerations are cost and purpose. Good art direction will be very helpful here. If it’s a question of opinion, you probably need to revisit your brief and get something more specific in mind. However, even a very specific brief could in theory use either one.
So what to consider?
Is the concept needing to be realistic or is it on something more abstract?
What I mean there is let’s say you’re making a blog post or a magazine editorial on leadership. Illustration can lend a lot more possibilities than photography. With photography you’re probably already picturing some generic stock photo of a person. Illustration can be all sorts of stuff. And especially when you delve into what about leadership the article is referencing it becomes even more likely that illustration is a better option.
Say it’s talking about leadership and how to communicate. What does that look like in photos? Pretty difficult to say. In illustration though you might see communication bubbles or tools for communication as objects around a person or in the background or any number of things.
On the other hand is the reality side. Are you showing a product? People probably don’t want to see an illustration of it but the real thing. (Or a photo-realistic rendering in some cases.)
Say you’re designing a cover for a product catalog. You probably want to use an assortment of photos of the products on the cover. Perhaps integrating some computer generated assets as well, but likely not solely an illustration.
The other big consideration is cost.
Hiring an illustrator has costs but will likely be less expensive than photography. Photography requires models, makeup artists, a photographer, sets, possibly travel costs, etc. This however does depend on the exact specs and project.
Emotion you mention. I haven’t seen any studies that really say definitively that one evokes more emotional response than the other. I haven’t even found studies saying one proves a higher response rate than the other. I think both forms have a lot of potential when done well, and both can also be really bad if done poorly. So ultimately the decision should be how grounded in reality do you need it to be and what your budget allows.
To make a hypothetical – if you want a futuristic piece with a space ship taking off for an advertisement for some mail delivery system. That sounds like it would be incredibly expensive to do well with photography (excluding using a random stock photo of a shuttle taking off). So illustration is probably the way to go.
On the other hand if you want to make a piece about a physical mailbox you’re manufacturing and trying to sell to homeowners you probably want a photo of it.
Some further reading:
One final note is that you don’t have to restrict yourself to one or the other. Focus on what’s going to communicate the art direction well and within budget. If that means combining both photographs and illustration than go for it.