Android :: Application Compatibility Between Different Versions

I can run my application on 1.5, 1.6 and 2.0 emulator and on a 1.5 device without any problems. But some of my users are reporting force close and strange menu entries (see the attachment) using 1.6 and higher version devices.
This screen should look like this:
http://www.taypo.com/blog/WindowsLiveWriter/TurkceKlavye0.7_6F1/settings_0.7_2.png

And the code that generates this screen is:
<?xml version="1.0" encoding="utf-8"?>
<PreferenceScreen xmlns:android="http://schemas.android.com/apk/res/android">
<PreferenceCategory android:title="@string/general_settings">
<CheckBoxPreference android:title="@string/haptic_feedback" android:defaultValue="true"android:summary="@string/haptic_feedback_summary" android:key="vibrate"/>
<CheckBoxPreference android:title="@string/sound_feedback" android:defaultValue="false" android:summary="@string/sound_feedback_summary" android:key="sound" />
<CheckBoxPreference android:title="@string/auto_caps" android:defaultValue="true"
android:summary="@string/auto_caps_summary" android:key="auto_caps" />
</PreferenceCategory>
<PreferenceCategory android:title="@string/layout_settings">
<ListPreference android:title="@string/keyboard_layout" android:defaultValue="TRQ" android:key="kbd_layout" android:entries="@array/layout_names" android:entryValues="@array/layout_values" />
</PreferenceCategory>
</PreferenceScreen>