The specifics of the heresy of the Nicolaitans seems to be that they believed it was OK to eat things sacrificed to idols, and that they allowed some heathen sexual practices (eg. participating in orgies, wife sharing) that were forbidden by the church.
Exegesis on the reference in D&C 117 seems to suggest that the term was being used to refer to a spirit of wanting to be in the church, but of the world. That is, having one's name on the role, but adopting the viewpoint and practices of worldly philosophy. It can be a difficult trap to avoid as we are influenced by the world we live in, and our view of what is acceptable is significantly affected by worldly philosophy. Worldly philosophy is not anchored to God and truth except inasmuch as imperfect people are dedicated to that principle. Even then, it is more like chasing truth, rather than being anchored to it. Even still, false doctrines exist in the church, penetrating from the outside, stemming from pride within the church, or stemming from errors. Worldiness and error exist within and hence, it is tempting to judge the difference between the church's doctrines and the currently popular philosophies of the world, as being solely attributable to errors within the church rather than our own errors and the errors of the world. Considering how the church is anchored to the rock of revelation, this would be a bad assumption.