I realise there are better forums to ask, but does anyone have experience of multilevel imputation using mice in R

[StealthPolarBear]

And be willing to answer some idiotic questions?

Some clustering code will compute k , number of clusters itself

Have you checked your method signatures in the documentation

cran.r-project.org/web/packages/mice/mice.pdf#page66

As often it will show default values for things you can set but don't have to

Ie you might call fnc(a, b)

And the code then does fnc(a, b, c=NULL)

And you can instead do fnc(a, b, c=6)

Professional statistician here, with experience of mice. I would caution against 'if it looks ok, accept it'. Rather, try to persevere to get a thorough understanding of exactly what the problem is. You need that understanding to get a solution. Use stackexchange and see if you can find a similar problem - chances are someone has already asked something very similar and there will be a solution that you can adapt for your purposes. But you have to really understand the issue first, so you know what words/phrases to search for on stackexchange (or indeed just google). Getting the language right and articulating the problem accurately is essential for anyone to be able to advise on what to do about it. Try subsetting your data and trying the same method with a small sample to help you understand what is going on; build your understanding with that first.

Also, I would say that multiple imputation in mice is not the best place for beginners - it is not a simple process by any means. It is an advanced technique.

Mumsnet is amazing. Women are amazing. That is all.

Yes, I've been using a small made up dataset first. I think it's not converging, that's why some NAs remain.
I'm now running it on a small portion of the real data and encountering other problems which I'm working through.
The main thing I'd love to understand is how to do the clustering thing. At the moment I'm just not giving that variable a method, but surely that just makes it passive rather than clustered.

ItJustKeepsGettingBetter I know you're right. But unfortunately this is the demand made of me

I'm not sure from the information given what the role of clustering is. Are you trying to impute a full dataset and then form clusters in that dataset? Or do you already have clusters identified in your dataset, and you want to impute by cluster? If it's the latter, then it's quite possibly multilevel multiple imputation that you need. See here as an example from stackexhange: stackoverflow.com/questions/38100892/imputation-using-mice-with-clustered-data

@PenfoldsFive I snorted my tea @ Numberwang

Amazing thread. I have no idea what's happening, but you are all marvelous.

Yes multilevel multiple imputation - that's it, thanks for the link and for confirming the search term.

Can you screenshot your code and explain step by step what you are trying to achieve? I have used the mice package to derive imputed data sets and then ran clustering algos on these. Are you trying to run regressions and get pooled estmates?

Does anyone know why my df contains -1 rather than the 1 I have selected? Why is it being turned negative?

Have you tried switching it off and back in again?

(And Hi, how the hell are you?)

Once I've done the imputations I want to get a proportion from the multiple imputed datasets

And if I select -1 it returns 1!
This must be something to do with the rodbc library, right?

To conduct multilevel multiple imputation, first ignore the clustering; run multiple imputation on the dataset ignoring clustering. Use that to learn exactly what PMM, logreg etc etc are. Solve all your problems in that process first, before attempting the multilevel version. Stepping up to the multilevel version will quite probably cause all sorts of headaches and you need to get to grips with how to solve problems in the less complex version before attempting multilevel. It's also entirely probable that the multilevel version will not be possible - depends on the number and nature of observations per cluster. It's rare to see multilevel multiple imputation work in practice. There are other R packages around (eg pan) that do multilevel multiple imputation, but best to stay with mice first and make sure you understand it. I would also say be prepared to go back to whoever is asking you do this and say 'the data doesn't support multiple multilevel imputation'

I've just started using R at Uni, so following this with interest. I'm a mature student, not good with maths but I love R so far...

Mombie you might be able to answer my screenshot! It's infuriating, surely it's about as simple as it ca be, why is it trying to negative everything?
Do I need t Google another way to access sql server?

Your negative df is because you have

This thread is going in my evidence bank for 'mumsnet knows the answer to EVERYTHING.'

Magpie complex I'm going to stalk you online until I find out where you live and then I am going to turn up on your doorstep and give you a huge kiss
Or I could just say THANK YOU

It's hard to know without being told what data it is or problem you are analysing but I think the person requesting this is potentially over complicating things. Can you push back and suggest a simpler approach?

Right now to look at that link and work out my cluster crap

One extra dash Magpie? You have convinced me to stick with SQL and Python!

I used R about 10 years ago, so I've forgotten most of it! I don't think I can help you much, sorry.
Do you know R has its own search engine (rseek.org/) - hopefully that will be useful for you.

I love this thread!!