Plus I think its unfair that some classes are bounded to strict conditions and some not. Why doesnt the artificer or wizard able to lose their powers then if the cleric or paladin does ?

I agree with you. If the player agrees to it, sure go ahead. As a bad surprise or a bad consequence of something else ? Find something that would affect anyone the same. Like jail.