bugAVR Downloader/UploaDEr - Bugs: bug #30559, Ft232 bit-bang support

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #30559: Ft232 bit-bang support

Submitter:  None
Submitted:  Tue 27 Jul 2010 02:06:47 AM UTC
   
 
Category:  None Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Fixed Privacy:  Public
Assigned to:  rliebscher Originator Name:  David Harris
Originator Email:  -email is unavailable- Open/Closed:  Closed
Release:  None Programmer hardware: 
Device type: 

Jump to the original submission

  Spam posted by oopasp3zine88
Sun 23 Nov 2014 01:49:04 PM UTC, comment #37: 

I guess patch #8419 solved the last remaining issue.

Rene Liebscher <rliebscher>
Group Member
Tue 03 Sep 2013 08:10:18 PM UTC, comment #36: 

I only see this problem when using libftdi1. With libftdi 0.20 this problem does not exist.

Maybe there is now a problem with threads in newer versions of libftdi.

Rene Liebscher <rliebscher>
Group Member
  Spam posted by mrlove2
Thu 13 Sep 2012 08:24:45 AM UTC, comment #34: 


> One little problem appeared however, now avrdude
> hangs after all the messages.


I've also observed this in some situations.  Will be fixed
before the next release.

Joerg Wunsch <joerg_wunsch>
Group administrator
Wed 12 Sep 2012 11:25:13 PM UTC, comment #33: 

Hi, sorry for the delay, I can confirm that the patch as of comment #32 indeed resolved the main problem I was having with os x and bitbang with Arduino.
Now the device responds correctly recognized etc. I haven't tested flashing something yet.

One little problem appeared however, now avrdude hangs after all the messages. The lst one says "avrdude: safemode: Fuses OK" and than nothing. It's necessary to force the process to terminate (ctrl-c).

I compiled latest trunk as of now, r1104.

Massimo <runaway>
Wed 15 Aug 2012 05:37:06 PM UTC, comment #32: 

Integrated patch for MacOS X at comment #30 in svn revision 1101.
Maybe this solves also the problems reported in comment #17.

Rene Liebscher <rliebscher>
Group Member
Sun 12 Aug 2012 02:01:13 AM UTC, comment #31: 

I managed to fix the baud rate problem, which I believe was caused by the BitBang Mode variable Pulse Width issue described in

http://www.ftdichip.com/Support/Documents/TechnicalNotes/TN_120_FT232R%20Errata%20Technical%20Note.pdf

My solution is to write each byte 4 times (and decimate input accordingly), but I'm a bit puzzled that nobody else has run into this problem, which presumably should affect any FT232R.

Matthias Neeracher <microtherion>
Thu 26 Jul 2012 11:45:00 PM UTC, comment #30: 

The reason this does not work on Mac OS X is that the sem_init family of calls is stubbed out in OS X: You can link to the calls, but they don't actually work.

I added a little patch below that emulates sen_init & co with libdispatch semaphores on OS X. Further more, the reader thread seems to need a pthread_testcancel on OS X, otherwise avrdude hangs at the end trying to cancel the thread.

With these changes, the mode works for me on OS X, although only at very slow rates – anything faster than 4800 baud leads to verification errors, and often reprogrammed fuses as well.

(file #26269)

Matthias Neeracher <microtherion>
Mon 11 Jun 2012 08:26:20 PM UTC, comment #29: 


> I thought Microsoft had added a "posix compatibility layer"
> to windows over a decade ago.


Well, AFAIK this is only an optional add-on, so its actual
value is questionable.  Second, threads are optional in the
Posix model, so you can obtain a UNIX branding without even
implementing threads.  The UNIX process model (with fork and
pipes and all that) is much more portable, and under Unix,
the difference between a thread and a process, performance-
wise, is neglicible.

> I'm not an "autoconf" hero.


Neither am I.  Nevertheless, it's not rocket science.  There's
already a good number of tests around in configure.ac which
could be used as templates to craft your own test.

Of course, the most preferrable option would be to use native
Win32 threads when compiling for a Win32 environment.  Win32
used to have threads for many years (because their processes
are fast as snails), and the subset of threading features
used here is not very large.  Perhaps some code for that
could be "borrowed" from the pthreads-win32 package without
requiring it being present as an external package?

Joerg Wunsch <joerg_wunsch>
Group administrator
Mon 11 Jun 2012 04:29:40 PM UTC, comment #28: 

I thought Microsoft had added a "posix compatibility layer" to windows over a decade ago. Apparently that compatiblity is not complete. :-(

The threads are only required to allow the ftdi to push data to the computer, even when the main avrdude process is mostly writing. This driver is fast because things are pipelined reading the feedback bit after every bit written is very timeconsuming.

 http://sources.redhat.com/pthreads-win32/

is the first hit for my google search for pthreads on win32.

I'm not an "autoconf" hero. If someone who DOES know what he's doing with autoconf is willing to help, please do!

Roger Wolff <rewolff>
Mon 11 Jun 2012 04:05:04 PM UTC, comment #27: 

Someone commented on the mailinglist that there is a pthread
implementation for Win32.  However, it's the responsibility
of configure.ac to probe for it then, and disable the
module if it's not around in the compilation environment.

Of course, it would generally be better to rewrite the code
to use Win32 native threads when running under Win32, unless
the prerequisite libftdi already requires pthreads anyway.

(One of the problems of the current implementation is that
the #include statement for pthreads.h is even outside the
check whether there's a libftdi at all.)

Joerg Wunsch <joerg_wunsch>
Group administrator
Mon 11 Jun 2012 03:39:41 PM UTC, comment #26: 

The current implementation breaks compilation for Win32
target systems, as it assumes the general availability
of pthreads:

make  all-recursive
Making all in .
mingw32-gcc -DHAVE_CONFIG_H -I.  -DCONFIG_DIR=\"/usr/local/etc\"  -Wall -Wno-pointer-sign -g -O2 -DWIN32NATIVE -MT libavrdude_a-ft245r.o -MD -MP -MF .deps/libavrdude_a-ft245r.Tpo -c -o libavrdude_a-ft245r.o `test -f 'ft245r.c' || echo './'`ft245r.c
ft245r.c:73:21: fatal error: pthread.h: No such file or directory
compilation terminated.
* Error code 1

Note that the "p" in "pthread" stands for "Posix", so it
should be clear it is not available in Win32 ...

Joerg Wunsch <joerg_wunsch>
Group administrator
Mon 13 Feb 2012 06:46:25 PM UTC, comment #25: 

libftdi 0.19
libusb 1.0.8
libusb-compat 0.1.3

Here the output of otool:
Max:Lucifer$ otool -L /opt/local/bin/avrdude
/opt/local/bin/avrdude:
/opt/local/lib/libusb-0.1.4.dylib (compatibility version 9.0.0, current version 9.4.0)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 635.19.0)
/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
/opt/local/lib/libusb-1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/opt/local/lib/libftdi.1.dylib (compatibility version 21.0.0, current version 21.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
/opt/local/lib/libreadline.6.2.dylib (compatibility version 6.0.0, current version 6.2.0)
/opt/local/lib/libncurses.5.dylib (compatibility version 5.0.0, current version 5.0.0)

Massimo <runaway>
Mon 13 Feb 2012 06:33:35 PM UTC, comment #24: 

Which version of libftdi (and libusb) do you use?

Rene Liebscher <rliebscher>
Group Member
Mon 13 Feb 2012 11:41:58 AM UTC, comment #23: 

@rewolf yep the first thing I checked was the connection. I'm using the same cable I used before with the old patch based upon libftd2xx wich worked.

Massimo <runaway>
Mon 13 Feb 2012 08:32:27 AM UTC, comment #22: 

On Sun, Feb 12, 2012 at 04:37:04PM +0000, Rene Liebscher wrote:

> (Using all places in the buffer does not work if you just use head
> and tail, because when it is full you can not decide if it is empty
> or full. Using a content counter would also help.)


FYI, I'm aware of this. I've moved from using bufsize-1 to bufsize
bytes in the buffer when I added the semaphore. The Semaphore is the
content-counter that you suggest.

Based solely on head and tail indeed you cannot decide wether the
buffer is full or empty. however when the read-thread blocks, the
buffer is full, when the mainprogram blocks the buffer is empty.

Using a content-counter is normally dangerous: both sides have to
write it. This would require a "critical section" in both threads.
Anyway, this is a problem I've delegated to the library by using the
semaphores.

As to the problem at hand.... It seems "suddenly" the FT232 comes up
with a big chunk of extra data. It shouldn't do this. It should
provide exactly (to the bit) the amount of data that you write to it.

You have made the proper connections between the FTDI programming pads
and the ICSP header?

Roger.

Roger Wolff <rewolff>
Sun 12 Feb 2012 10:51:26 PM UTC, comment #21: 
Massimo <runaway>
Sun 12 Feb 2012 10:28:57 PM UTC, comment #20: 

Let's try to add some debugging code (see attached file) so we can see what is sent and comes back.

(file #25051)

Rene Liebscher <rliebscher>
Group Member
Sun 12 Feb 2012 04:53:08 PM UTC, comment #19: 

@rliebscher The buffer overflow message is indeed gone, however avrdude still hangs. Here the new output: http://pastebin.com/vnkPXgb9
(I tested with the normal serial port and it respond correctly with the signature)

Massimo <runaway>
Sun 12 Feb 2012 04:37:03 PM UTC, comment #18: 

Could you please change line 492 in ft245r.c from:

    sem_init (&buf_space, 0, BUFSIZE);

to

    sem_init (&buf_space, 0, BUFSIZE-1);

and check if this changes anything.
(Using all places in the buffer does not work if you just use head and tail, because when it is full you can not decide if it is empty or full. Using a content counter would also help.)

This should remove the "buffer overflow. Cannot happen!" message.

But it is not clear why you get this on MacOS as I never saw it on Linux or Windows.

Rene Liebscher <rliebscher>
Group Member
Sun 12 Feb 2012 01:31:15 PM UTC, comment #17: 

Hi, I'm trying to use ftdi bitbanging with my arduino deuemilanove (ATmega328P) with latest avrdude trunk (rev. 1068) with my MacBook Pro and Os X Lion.
However the arduino do reset but then doesn't respond (I unloaded the FTDI kext).
Here my output: http://pastebin.com/p1FjuzS3

Massimo <runaway>
Sun 05 Feb 2012 09:10:39 PM UTC, comment #16: 

Ooops Cancel that. I was using the STK500V2 programmer. Not the syncbb one.

Roger Wolff <rewolff>
Sun 05 Feb 2012 09:03:01 PM UTC, comment #15: 

Rene, I think I see the progress bar do bad things. I'm probably passing the wrong arguments to the progress bar function.

If you already know what the progress bar function should be passed, I'm hoping it's a 2 minute fix for you. If not, let me know and I'll look in to it.

(Symptoms: progress bar progresses slowly that you're thinking: that'll take a while, and then suddenly it's done. It seems like only 10% of the progress bar is used).

Roger Wolff <rewolff>
Sun 05 Feb 2012 08:00:34 PM UTC, comment #14: 

Tested with Windows XP (MinGW) and Ubuntu Linux 11.10.
Integrated in svn revision 1066.

Rene Liebscher <rliebscher>
Group Member
Sun 05 Feb 2012 02:08:22 PM UTC, comment #13: 

I've added two lines that should cancel and wait for the reader thread. No apparent badness under Linux. Rene, can you test this under Windows? Does the sigfault go away?


(file #24986)

Roger Wolff <rewolff>
Sun 05 Feb 2012 09:59:25 AM UTC, comment #12: 

Just for the record.

I don't think that merging them should be done "just because they use similar chips".

If merging is done I think the reason should be: "Because they share a lot of code".

For example, suppose the rasbperry pi board has an SPI port (it does), then AVRDude might be able to program AVRs through that SPI port.

But suppose that the board also has general purpose IO pins (it does), then AVRDude may be able to program AVRs through those GPIO pins. In fact, the SPI pins can be configured and used as GPIO pins as well. So it can be said that the same chip/board can switch between the hardware SPI and software bitbanging on the same pins.

I think that conceptually they are different drivers and should not be merged into one sourcefile.

A very similar thing is happening here: The mpsse module in the modern FTDI chips can be used (one driver) or a different "module" of the chip (syncbb) can be used (the other driver).

And just like the example of the raspberry pi board, the MPSSE mode cannot allow pin assigments to be "random", while the bitbanging modes allow (almost?) every combination of assignments of pins to functions.

Roger Wolff <rewolff>
Wed 01 Feb 2012 10:45:31 PM UTC, comment #11: 

I added this in svn revision 1055.

The name used in the config file is ftdi_syncbb. But files are still named ft245r.

The reason for the difference is the fact that now the pin numbers start from 1 (as avrftdi and others do), where ft245r used numbers starting from 0. But 0 is also used internally for not set pin numbers, so if extending the programmer to support  leds, it would not be clear if it is not set or should use pin 0.

There are now some new programmer entries : ft232r, ft245r, bwmega, arduino-ft232r (and diecimila as its alias, see also http://www.geocities.jp/arduino_diecimila/bootloader/index_en.html ).

I think it should be later merged into avrftdi, so it detects the chip, checks the pin configuration and uses then MPSSE or does bitbanging, if MPSSE is not usable, because not available on the chip or signals are not assigned to the MPSSE pins.

Rene Liebscher <rliebscher>
Group Member
Sat 21 Jan 2012 08:58:12 PM UTC, comment #10: 

I did some cleanup and some smaller changes (you can now specify bitclock -B in us or baudrate -b in baud, the previous patch misused the bitclock option as baudrate.).

It does talk now to an ardunio mega (ftdi signals of X3 connected to the ISP header) and BitWizard ftdi_atmega board [kindly provided by Roger ;-) ] I tried Windows XP and Ubuntu 11.10. On windows it needs the libusb driver installed for the devíce.

However, reading the arduino I got a segmentation fault at program exit. This might be a problem with the still running thread.

(file #24864)

Rene Liebscher <rliebscher>
Group Member
Fri 20 Jan 2012 05:36:57 PM UTC, comment #9: 

Hmm. I fixed up a new version a few days ago and I thought I had posted it. Maybe it was the SOPA protest that prevented me from posting it here.

Anyway, cleaner patch without serjtag is attached.

(file #24855)

Roger Wolff <rewolff>
Mon 16 Jan 2012 08:44:13 PM UTC, comment #8: 

I think we should left out serjtag, until anybody comes up with references what this programmer is and is willing to test it.

Rene Liebscher <rliebscher>
Group Member
Mon 16 Jan 2012 06:13:25 PM UTC, comment #7: 

Rene, thank you for checking things out (compile test) on Windows! I forgot about that part...

I have now also patched serjtag (where I don't know where the hardware comes from.... I have tried googling if maybe I can make the hardware on short notice, but no such luck. It is simply mentioned on the pages that link to the patches that I have ported here.)

My guess is that serjtag was copied by someone into ft245r, and then modified there. I guess the patches could be split: the serjtag driver should be considered less tested as I don't have the hardware to test it. (nor can I find documentation that tells me about this).

Rene, can you also perform functional tests?

I have removed the "report progress" calls from the programming loops. This is apparently now handled outside the loop in the driver.

Both ft245r.c and serjtag.c have "generic" paged_write functions that are used for eeprom writing. I think I saw some coding errors in that part and wondered how it could have passed my testing I claimed I had done below. The answer is that those are called (in my testing) with m->paged set to false, so that code was never executed. The question is: Can non-flash memory in AVRs be paged?

If not, that would allow me (us) to remove some error-prone (and for me difficult to test) code.....

Attached is the latest version and this time generated with svn diff.... (My understanding of svn has moved from cut-pasting examples found on the web to: I know two commands! :-) (they are: svn diff and svn add) ).




(file #24840)

Roger Wolff <rewolff>
Fri 13 Jan 2012 10:03:41 PM UTC, comment #6: 

I get following warnings and errors compiling with MinGW:


serjtag.c: In function 'serjtag_initpgm':
serjtag.c:706:23: warning: assignment from incompatible pointer type [enabled by default]
serjtag.c:721:20: warning: assignment from incompatible pointer type [enabled by default]
serjtag.c:722:19: warning: assignment from incompatible pointer type [enabled by default]

The last two are again the missing addr parameter at the paged_(load|write) functions.

ft245r.c: In function 'ft245r_open':
ft245r.c:516:3: error: 'r' undeclared (first use in this function)
ft245r.c:516:3: note: each undeclared identifier is reported only once for each function it appears in
ft245r.c:516:3: warning: implicit declaration of function 'FT_Open' [-Wimplicit-function-declaration]

Why not use the libftdi function to open the device. It works for  avrftdi, so it should do here.

BTW: For creating the patch use 'svn diff' (don't forget to add new files first 'svn add serjtag.* ft245r.*') This makes the diff only for the repository files and ignores all in the build process created files.

Rene Liebscher <rliebscher>
Group Member
Thu 12 Jan 2012 04:12:07 PM UTC, comment #5: 

Attached is the patch that compiles, builds and verifies against the SVN codebase....

avrdude-5.11.1 users beware:
(The previous patch might have non-functional eeprom reading/writing. )


(file #24807)

Roger Wolff <rewolff>
Thu 12 Jan 2012 01:44:15 PM UTC, comment #4: 

Hi,

please you use the latest svn revision as base for your patch.

Also the interface of the *_paged_(load|write) function has changed, so this must be adapted. (You may have a look at last changes of avrftdi.c, which needed similar changes. http://svn.savannah.nongnu.org/viewvc/trunk/avrdude/avrftdi.c?root=avrdude&view=log )

Rene Liebscher <rliebscher>
Group Member
Thu 12 Jan 2012 01:30:49 PM UTC, comment #3: 

Dear avrdude maintainers,

I'm going to attach a patch that uses the FT2xxR synchronous bitbang modes to program the AVR chips. This can be done QUITE fast. I got over 2k/sec programming+verifying yesterday. I don't understand what's changed since yesterday, but I only  get just below 2k/sec today. :-( (this means over 4k/sec programming only).

I THINK this throughput can be increased even further but I haven't been able to get this to work just yet.

Previous versions of this "patch" relied on the non-free ftd2xx library from FTDI. This one works with libftdi.

I am not able to test it under windows.

The reads from the device have to be asynchronous under Linux. I therefore had to add a dependency on pthreads.

Otherwise, it seems a deadlock would develop: while sending a datablock to the FTDI, it needs to return the synchronous "return data". When the application hasn't issued the read yet, the kernel will prevent the device from sending the data, and then when the buffers in the ft2xxr fill up, it will also stop accepting data, blocking the write....

I have found that there is also a new avrftdi programmer in the current release. This programmer works for ft2232(h) chips and refuses to work for the ft232r that I have here.

(some) Arduino users will be able to download their sketches 4-10 times faster with this patch. :-)


(file #24806)

Roger Wolff <rewolff>
Wed 08 Jun 2011 03:47:34 PM UTC, comment #2: 

Joerg,

The FTDI is a funny chip. It will output the "bitbang" bytes (you always give it a full byte, even if you're changing just one bit), at any rate you want. So the patch that's available (I'll find you an URL later on). will adapt the setting of the FTDI chip to whatever bitrate you spedcify with -B.

It programs my atmega168 about 10x faster than my 19200 baud STK500V2 compatible programmer (which is based on the same atmega168, which due to clock division reasons doesn't work reliably above 19200).

So NO, I don't think bitbanging is slow.

I have a board where the FTDI control lines are permanently connected to the programming pins of my atmega on the board. With Arduino you'll have to make a small patch cable from the ftdi-control-pins-header that some boards have to the AVR-programming header.

Here is a page with a "howto" that worked for me. It links to the patch...
http://www.geocities.jp/arduino_diecimila/bootloader/index_en.html

Roger Wolff <rewolff>
Tue 27 Jul 2010 05:25:20 AM UTC, comment #1: 

Please have a look at patch #4330, and tell us whether this
is what you had in mind.  If so, please give that patch a
try, and leave us some feedback about whether this is working
for you or not.

Joerg Wunsch <joerg_wunsch>
Group administrator
Tue 27 Jul 2010 02:06:47 AM UTC, original submission:  

I understand that bit-banging is slow.  However, it is useful to bootstrap a processor with a bootloader.  We have a product with hardware supper for this with one pin-header to close this option (LEDuino).  Can I request native support for this in AVRDUDE.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #26269:  macosx.patch added by microtherion (847B - application/octet-stream - Fixes hangs on Mac OS X)
file #25051:  ftdibb_debug.patch added by rliebscher (2KiB - application/octet-stream)
file #24986:  cancel_join.diff added by rewolff (549B - text/x-patch)
file #24864:  ft245r.patch added by rliebscher (28KiB - application/octet-stream)
file #24855:  bw_ft245only_svn.patch added by rewolff (28KiB - text/x-diff)
file #24840:  bw_ft245_svn2.patch added by rewolff (47KiB - text/x-diff)
file #24807:  bw_ft245_svn.patch added by rewolff (62KiB - text/x-diff)
file #24806:  bw_ft245r.patch added by rewolff (56KiB - text/x-diff)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by oopasp3zine88 (Posted a comment)
  • -email is unavailable- added by mrlove2 (Posted a comment)
  • -email is unavailable- added by microtherion (Updated the item)
  • -email is unavailable- added by runaway (Posted a comment)
  • -email is unavailable- added by rliebscher (Posted a comment)
  • -email is unavailable- added by rewolff (Posted a comment)
  • -email is unavailable- added by rewolff
  • -email is unavailable- added by joerg_wunsch (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 15 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2014-11-23 rliebscher StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2012-07-26 microtherion Attached File- Added macosx.patch, #26269
    2012-02-12 rliebscher Attached File- Added ftdibb_debug.patch, #25051
    2012-02-05 rewolff Attached File- Added cancel_join.diff, #24986
    2012-02-01 rliebscher StatusIn Progress Ready For Test
    2012-01-21 rliebscher Attached File- Added ft245r.patch, #24864
        StatusNeed Info In Progress
        Assigned toNone rliebscher
    2012-01-20 rewolff Attached File- Added bw_ft245only_svn.patch, #24855
    2012-01-16 rewolff Attached File- Added bw_ft245_svn2.patch, #24840
    2012-01-12 rewolff Attached File- Added bw_ft245_svn.patch, #24807
    2012-01-12 rewolff Attached File- Added bw_ft245r.patch, #24806
    2011-06-08 rewolff Carbon-Copy- Added -email is unavailable-
    2010-07-27 joerg_wunsch StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-cf05.
    Corresponding source code