Re: [microblaze-uclinux] Microblaze with MMU, loadable module printk problems

[jeffrey.fuller@xxxxxxxxx]

John,

Your work-around fixed our problem and
we appear to be up and running with loadable modules. Thanks very much
for the help. If you can remember, please let us know what Xilinx says
about this issue.

Regards,
Jeff

Jeff Fuller
Software Engineer
EI WW SOFTWARE SYSTEMS ENGINEERING

Eastman Kodak Company
2400 Mt Read Blvd.
Rochester, NY 14650-3019

jeffrey.fuller@xxxxxxxxx
Office: (585)726-6908
www.kodak.com

John Williams <jwilliams@xxxxxxxxxxxxxx>
Sent by: owner-microblaze-uclinux@xxxxxxxxxxxxxx

05/12/2008 09:47 PM

Please respond to
microblaze-uclinux@xxxxxxxxxxxxxx

To	microblaze-uclinux@xxxxxxxxxxxxxx
cc
Subject	Re: [microblaze-uclinux] Microblaze with MMU, loadable module printk problems

Jeff,

I've confirmed your report and discovered the cause - as a workaround 
can you please comment out the following line in

arch/microblaze/kernel/module.c, around line 100

                 case R_MICROBLAZE_64:
                    
    old_value = ((location[0] & 0x0000FFFF) << 16)
|
                    
                    (location[1]
& 0x0000FFFF);
                    
    //value += old_value;
^^^^^ COMMENT OUT THIS LINE ^^^^^^^^^
                    
    location[0] = (location[0] & 0xFFFF0000) |

and give it a try?

With this fixup I get the correct module behaviour, both the printk()s
and /proc entry.

The reason for this is a change between how mb-gcc handles ELF 
relocations from the previous (pre-mmu) to new (post-mmu) versions.

This issue has caused various amounts of pain over the years, and I'm 
sorry to see it return.

The "value += oldvalue" code was previously required because
the 
compiler/linker split relocations between the code itself (embedding 
non-zero offsets in the opcodes) and the ELF relocation table (typically
the offset of the .rodata section, for example).

Because of this splitting, we had to crack open the opcode from .text,
add in the offset from the reloc table, then munge it back into .text to
generate the fixed relocation.

However, now it seems that the complete relocation is stored in the 
reloc table.  This would be great, *if* the compiler inserted zero
offsets in the opcodes.  But it doesn't - they're still in there -
grrr!

So, we have to just ignore the "oldvalue", and put the new (reloc)
value 
straight in.

The problem is that this is compiler dependent, I don't like the idea of
an #ifdef on compiler version in module.c

I'll have a chat to the toolchain folks at Xilinx and see about the best
way forward.  Ideally we can just have the zero value offsets put
into 
the opcodes, that way no workaround will be required.  But, maybe
there's some gcc/ld thing I'm missing that makes this the Right Way to
do things.

Thanks for your patience and helpful test case.

John


jeffrey.fuller@xxxxxxxxx wrote:
> 
> John, thanks for the help. I had erred in saying that the driver would
> not do printks correctly on non-MMU Microblaze. The module you sent
> prints correctly and so do ones I build (long story short: I fooled
> myself into thinking the same problem was on both non-MMU and MMU).
> 
> However, we do still have a problem on Microblaze with MMU, and today
we 
> discovered something interesting by accident. This can be recreated
with 
> the following code and method:
> 
> CODE:
> ============
> #include <linux/module.h>
> #include <linux/kernel.h>
> #include <linux/errno.h>
> #include <asm/uaccess.h>
> #include <linux/fs.h>
> #include <linux/proc_fs.h>
> 
> #define STRING1 "1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
19 20\n"
> #define STRING2 "a b c d e f g h i j  k  l  m
 n  o  p  q  r  s  t \n"
> 
> static int __init
> NspbInit(void)
> {
>     printk("%s", STRING1);
>     printk("%s", STRING2);
>     return 0;
> }
> 
> module_init(NspbInit);
> 
> static void __exit
> NspbCleanup(void)
> {
> 
> }
> 
> module_exit(NspbCleanup);
> ============
> 
> METHOD:
> Note: You must run this on a Microblaze w/ MMU in order to duplicate:
> 
> 1. Build the above module with the mb-linux-gcc compiler (as I 
> understand it, this is the compiler that should be used for Microblaze
> w/ MMU).
> 
> 2. insmod the resulting module.
> 
> 3. Expected output:
> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
> a b c d e f g h i j  k  l  m  n  o  p
 q  r  s  t
> 
> 4. Actual output:
>  3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
> 
> 5. Now, build the above module with the microblaze-uclinux-gcc (as
I 
> understand it, this is the compiler that should be used for NON-MMU
> Microblaze).
> 
> 6. insmod the resulting module.
> 
> 7. You get the expected output.
> 
> So, in conclusion, when building a module for Microblaze w/ MMU and
> using the mb-linux-gcc compiler, which is the compiler we thought
should 
> be used, we get incorrect results. When using the 
> microblaze-uclinux-gcc, we get expected results. Again, to reiterate,
> this is all on Microblaze w/ MMU. One further note: in our experience,
> just about any combination of printks and strings (including, for
> example printk("This is a string\n") should have problems
when compiled 
> with mb-linux-gcc, though we have seen a few simple, single strings
> print correctly.
> 
> Can anyone tell me if we've somehow got our understanding of the 
> compilers mixed up? Or is there a real bug here? Can anyone try the
> above test code and method on a Microbaze MMIU? Any help would be
> greatly appreciated.
> 
> Regards,
> Jeff
> 
> *Jeff Fuller*
> Software Engineer
> EI WW SOFTWARE SYSTEMS ENGINEERING
> 
> Eastman Kodak Company
> 2400 Mt Read Blvd.
> Rochester, NY 14650-3019
> _
> __jeffrey.fuller@xxxxxxxxxx <mailto:jeffrey.fuller@xxxxxxxxx>
> Office: (585)726-6908 _
> __www.kodak.com_ <http://www.kodak.com/>
> 
> 
> 
> *John Williams <jwilliams@xxxxxxxxxxxxxx>*
> Sent by: owner-microblaze-uclinux@xxxxxxxxxxxxxx
> 
> 05/11/2008 07:38 PM
> Please respond to
> microblaze-uclinux@xxxxxxxxxxxxxx
> 
> 
>                  
> To
>                  microblaze-uclinux@xxxxxxxxxxxxxx
> cc
>                  
> Subject
>                  Re:
[microblaze-uclinux] Microblaze with MMU, loadable module printk 
> problems
> 
> 
>                  
> 
> 
> 
> 
> 
> Hi Jeff,
> 
> jeffrey.fuller@xxxxxxxxx wrote:
> 
>  > An update on this problem. I tried running with v0.30 of
the Petalinux
>  > distro uclinux (non-MMU microblaze) and got the same problem.
I also
>  > modified the rng-core.ko module (random number driver)
included in the
>  > Petalinux Microblaze with MMU v0.10 to include some printks,
then used
>  > make "menuconfig" to turn the build of the module
on. I then loaded the
>  > resulting module using insmod. The output of the printks
was incorrect.
>  >
>  > So, what I am seeing so far is that I am unable to get
reliable output
>  > from loadable modules running on Microblaze. This includes
modules that
>  > I didn't write and that I didn't compile (i.e. makefiles
for the
>  > rng-core.ko module were part of the distro). Has anyone
else actually
>  > seen valid printk or /proc output from a loadable module
running on
>  > Microblaze? My next step might be to compile my driver
into the kernel
>  > and see if printks work then.
> 
> Here's what I get on a board here (noMMU) running petalinux-v0.30-rc1
> (with a few additions, but nothing that should relate to modules).
> 
> # insmod ./modtest.ko
> Using ./modtest.ko
> modtest: module license 'unspecified' taints kernel.
> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
> a b c d e f g h i j  k  l  m  n  o  p
 q  r  s  t
> 
> All looks fine.
> 
> Here are my busybox insmod configs:
> 
> CONFIG_USER_BUSYBOX_INSMOD=y
> CONFIG_USER_BUSYBOX_LSMOD=y
> CONFIG_USER_BUSYBOX_MODPROBE=y
> CONFIG_USER_BUSYBOX_RMMOD=y
> # CONFIG_USER_BUSYBOX_2_2_MODULES is not set
> CONFIG_USER_BUSYBOX_2_4_MODULES=y
> CONFIG_USER_BUSYBOX_2_6_MODULES=y
> # CONFIG_USER_BUSYBOX_INSMOD_VERSION_CHECKING is not set
> CONFIG_USER_BUSYBOX_CHECK_TAINTED_MODULE=y
> # CONFIG_USER_BUSYBOX_INSMOD_LOADINKMEM is not set
> CONFIG_USER_BUSYBOX_INSMOD_KSYMOOPS_SYMBOLS=y
> # CONFIG_USER_BUSYBOX_INSMOD_LOAD_MAP is not set
> 
> I've attached the module binary I built and loaded - maybe you can
give
> it a try on your board (noMMU).  CPU settings are
> 
> CONFIG_XILINX_MICROBLAZE0_USE_MSR_INSTR=1
> CONFIG_XILINX_MICROBLAZE0_USE_PCMP_INSTR=1
> CONFIG_XILINX_MICROBLAZE0_USE_BARREL=1
> CONFIG_XILINX_MICROBLAZE0_USE_DIV=1
> CONFIG_XILINX_MICROBLAZE0_USE_HW_MUL=1
> CONFIG_XILINX_MICROBLAZE0_USE_FPU=0
> 
> yours will need to match or it will not work - I can rebuild with
a
> subset if you like.
> 
> I'll test it on an MMU boot when I get the chance - Michal maybe you
can
> give it a try if you have one handy?
> 
> Regards,
> 
> John
> 
> 
> 
> [attachment "modtest.ko" deleted by Jeffrey J. Fuller/470335/EKC]
> 

___________________________
microblaze-uclinux mailing list
microblaze-uclinux@xxxxxxxxxxxxxx
Project Home Page : http://www.itee.uq.edu.au/~jwilliams/mblaze-uclinux
Mailing List Archive : http://www.itee.uq.edu.au/~listarch/microblaze-uclinux/