The Tories say the Home Office appears "incapable" of keeping data secure and criminals may seek compensation.
Details of 84,000 prisoners in England and Wales were lost by private firm PA Consulting. The Home Office said a full investigation was being conducted.
The information commissioner's office described it as "deeply worrying".
PA Consulting has searched its premises and looked at CCTV recordings in an attempt to recover the missing memory stick - a commonly used portable storage device for computer files. It is not clear how it came to be lost.
'Horrified'
The missing device includes un-encrypted details about 10,000 prolific offenders.
It also includes the including names, dates of births and some release date of all 84,000 prisoners in England and Wales - and a further 33,000 records from the police national computer.
There are fears offenders could now sue the Home Office over the loss of the data.
PA Consulting was handed data as part of a research project on tracking offenders through the criminal justice system.
A Home Office spokesman said the data had been "held in a secure format on site and downloaded onto a memory stick for processing - which has since been lost".
Government departments were ordered to tighten up their security procedures after the loss of two discs containing personal details of every child benefit claimant in November.
But shadow home secretary Dominic Grieve said the latest loss of data showed the Home Office was "entirely incapable of keeping it secure".
'Rash of losses'
And he said there was a serious risk that if criminals' details were found by a third party, they could sue the government for compensation.
"It is ultimately the Home Office's responsibility to maintain the security of this material," he added.
PA Consulting is one of the companies that has been involved in developing the government's controversial ID card scheme - opposed by the Conservatives and Liberal Democrats.
Lib Dem leader Nick Clegg said: "The government will no doubt seek to blame private contractors, but the rash of data losses over the last two years confirm that there is something much more worrying at stake: this government cannot keep any information safe."
David Smith, Deputy Commissioner in the Information Commissioner's Office, said the latest loss showed that personal information could be a "toxic liability" if not handled properly.
"It is deeply worrying that after a number of major data losses and the publication of two government reports on high profile breaches of the Data Protection Act, more personal information has been reported lost," he said.
Adequate safeguards
He said his office would decide "what further action may be appropriate" once it saw a copy of the report on the internal investigation.
Labour MP and chairman of the home affairs select committee Keith Vaz told BBC Radio 4's Today programme he hoped the government had put adequate safeguards in place.
"If you hand out memory sticks almost like confetti to companies and ask them to do research for you, then you have to be absolutely certain... that the company concerned has put in practice procedures which will be just as robust as the procedures that I hope the government has followed," he said.
A spokesman for PA Consulting refused to comment on the data loss.
A Home Office spokesman said: "A full investigation is being conducted. Police and the Information Commissioner have been informed."
Earlier this month the BBC apologised after a memory stick containing the personal details of hundreds of children who had applied to take part in a TV show was stolen from a vehicle.
On Tuesday, a BBC analysis found sensitive data potentially affecting more than four million people had been lost by government departments in the year to April.
BBC
| Buying | Selling | |
| Euro | 1.9865 | 1.9961 |
| Dolar | 1.5711 | 1.5787 |
| Sterlin | 2.3159 | 2.3280 |













