ATHENS, Greece — Greece's neo-Nazi party Golden Dawn took a blow this past week with its leaders detained pending trial, but the government's goal to eradicate it may not happen anytime soon.

Rights advocates warned of a long legal process against two dozen defendants and media questioned the conditional release of several of them, fearing some cases may be thrown out.

Public opinion polls also showed that the party, despite losing a few percentage points, still had enough support to remain in parliament.

Golden Dawn's founder and leader Nikos Michaloliakos was charged with heading a criminal organization, and was taken to a high-security prison in Athens last week along with his second-in-command and one of his lawmakers pending their trials.

Magistrates have compiled a large file on Golden Dawn members that reportedly includes attempted homicides, robberies and an arson attack, in addition to the fatal stabbing of hip-hop artist Pavlos Fyssas allegedly by a self-confessed neo-Nazi last month.

Michaloliakos and his colleagues each face a sentence of at least 10 years in prison if convicted.

It was a dramatic reversal of fortune for the pugnacious group which had previously been seen as allowed to insult and attack opponents with impunity while boosting its ratings to the point of becoming Greece's third most popular party.

Conservative Prime Minister Antonis Samaras said authorities had "crushed the neo-Nazis" and that their influence "is falling precipitously as we speak."

"I am confident that populism will not gain further ground. It will lose popular support," Samaras said on Thursday at the close of a five-day visit to the United States.

An opinion poll by the VPRC firm released Friday showed that 68 percent of respondents agreed that the group was a "criminal organization."

And though the party suffered a six-percent drop in ratings to 8.5 percent in the poll, it was still enough to keep Golden Dawn in third place, and safely in parliament.

However, the Hellenic League for Human Rights said the judicial process will be "neither simple nor fast."

"Greek politics and justice cannot very well tell every citizen who joined this party 'sorry, this was a criminal organization, here's 10 years in prison,'" the league said in a statement.

Given the nature of the charges, it could take a year to bring the case to trial, justice sources noted.

"The problem is that Greek justice had not investigated neo-Nazi violence before, so now it must tread carefully to assure a fair process," said Dimitris Christopoulos, a law professor and vice-president of the league.

PM Samaras has pledged to "eradicate" the party, which picked up more than 420,000 votes in the last election.

"It is a shame to watch a neo-Nazi group gaining ground in a country, which only half a century ago, lost 10 percent of its population during the Nazi occupation," he said.

To do so, however, Samaras and his ministers must also tackle Golden Dawn's suspected support among many members of the armed forces and police.

Electoral data showed that a disproportionately large number of police voted for Golden Dawn last year, and there are reports that army reservists helped the group conduct clandestine military-style training.

"We are adamant in our target to clean-up the Greek police from any racist elements," Citizen Protection Minister Nikos Dendias told CNN in an interview.

But the league of human rights noted: "The same police ministry that claims a leading role in the cleanup is the same . . . that failed to protect victims" of racist attacks.

Samaras is under pressure from his European partners to deal with the neo-Nazi menace, which could cast a pall on Greece's upcoming stint as EU president starting January.

The authorities have pledged to stiffen the law on state funding for parties and there are calls to ban Golden Dawn, but experts have warned against measures that could set a negative precedent.

"Legislation should not be promulgated on the hoof, that would be shocking," said Nikos Alivizatos, a professor of constitutional law.

Part of this caution is addressed to the media, which lambasted judicial authorities after three prominent Golden Dawn lawmakers were conditionally released pending trial.

"This is a disgrace to the operation of democracy," leftist Eleftherotypia daily wrote while weekly Pontiki added: "The snake is still in our bosom."

Formerly on the fringe of Greek politics, Golden Dawn has skyrocketed to popularity by tapping into widespread anger over unpopular reforms in a country that is currently slogging through its sixth year of recession and where unemployment among the young stands at a staggering 60 percent.

The party denies the charges leveled against it, and claims it is the victim of a smear campaign ahead of next year's local elections.