COVID-19 pandemic has affected every country across different continents, be a developed or developing economy. The COVID-19 pandemic has led to a dramatic loss of human life worldwide and presents an unprecedented challenge to public health, food systems and the world of work. Conducting evaluation during COVID-19 pandemic was even more challenging as compared to the evaluation in conflict areas. Sudden lockdown and sustained restrictions was unexpected and affected the evaluators plan of actions for the ongoing as well as forthcoming evaluation activities. Not only primary data collection but secondary research also got hampered as access to knowledge resource centres/libraries stopped due to closure of these centres. As far as primary data collection is concerned, not only data collection exercise got stopped but even for those evaluations where data collection had been completed, the electronic data entry of filled-in survey schedules got stalled for a while. The paper discusses the critical components of evaluation, which gets affected during pandemic like situation such as use of participatory evaluation techniques; missing evidence based policy decisions; external and internal validity not ensured or ethical norms get compromised. To overcome such situations, the evaluation world should be ready with the suggested solutions such as, Use of Artificial Intelligence, computer-assisted interviews, capacity building of community members for participatory evaluation and making ethical review of evaluation protocols mandatory.