When Adagio Payables and MultiCurrency is set up correctly for rate types and exchange rates, it does not matter whether the transaction is for invoices or for payments, the operation of divide or multiply is consistant and works as expected. I believe something may not be set up correctly in either AP or MC which is causing what you are experiencing.

Because I cannot duplicate what you have posted regarding a different rate operation for invoices and checks, I'll need more information and an example of what you did and how things are set up. I'd need to know:

What rate type was used for invoices and what the rate operation of that Rate table is (either multiply or divide)? Look at the invoice posting journal for that US Vendor.
What rate type was used for payment and the associated rate operation? Look at the posting journal for that payment.
What was the exchange rates for the invoice and the payment? Both can be found on respective posting journals.
What was the invoice amount (in US dollars) and the calculated home equivalent, from the posting journal?
What was the payment amount (in US dollars) and the calculated home equivalent, from the posting journal?