Patch #3637 is online...

I've received an email from caml that the OCAML issue with the open file descriptors (id = 3423)  has been fixed in the CVS version of OCAML.

I think I've verified that the problem is in the OCAML unix library, not in mldonkey.  I modified OCAML's unix library and moved the unix_error call that was in alloc_sockaddr to the routines that called alloc_sockaddr and returned a value of -1 from within alloc_sockaddr.  That allowed me in unix_accept to close(retcode) before calling unix_error.  I rebuilt 2.5.29ab with the begin/end added to tcp_handler, but removed the "close t" because that is incorrect.  The log file indicates that I've gotten a number of tcp_handler errors but now I have only one orphaned file descriptor (the one due to send_paquet_to_mlchat and I've verified that I still have a high ID (since the close t was removed).

It is probably a good idea to continue to trap the error in the tcp_handler routine so mine now looks like:

let tcp_handler t sock event =
  match event with
  | CAN_READ
  | CAN_WRITE ->
     begin
     try
       let s,id = Unix.accept (fd sock) in
       if !verbose_bandwidth > 1 then lprintf "[BW2 %6d] accept on %s\n" (last_time ()) t.name;
       (match t.accept_control with
          None -> () | Some cc ->
             cc.nconnections_last_second <- cc.nconnections_last_second + 1);
       incr nconnections_last_second;
       t.event_handler t (CONNECTION (s,id))
 with e ->
    lprintf "Exception tcp_handler: %s\n" (Printexc2.to_string e);
    raise e
    end
  | _ -> t.event_handler t (BASIC_EVENT event)

although this doesn't fix the problem, it does keep other lines of code from executing when an error occurs.

The final orphaned file descriptor can be fixed by changing send_paquet_to_mlchat in src/daemon/common/commonChat.ml to close chan_out on a connect error.  My routine looks like:

let send_paquet_to_mlchat (p : C.packet) =
  let domain = Unix.PF_INET in
  let sock = Unix.socket domain Unix.SOCK_STREAM 0 in
  let inet_addr =
    let host = !!O.chat_app_host in
    try Unix.inet_addr_of_string host
    with _ ->
        let h = Unix.gethostbyname host in
        h.Unix.h_addr_list.(0)
  in
  let sockaddr = Unix.ADDR_INET (inet_addr, !!O.chat_app_port) in
  let chanout = Unix.out_channel_of_descr sock in
  try
    Unix.connect sock sockaddr;
    Chat_proto.write_packet_channel chanout p;
    flush chanout;
    close_out chanout
  with
  | Unix.Unix_error (e,s1,s2) ->
      let s = (Unix.error_message e)^" : "^s1^" "^s2 in
      lprintf "%s\nchat_app_host=%s chat_app_port=%d\n" s
        !!O.chat_app_host !!O.chat_app_port;
      close_out chanout
  | e ->
      lprintf "%s\nchat_app_host=%s chat_app_port=%d\n"
        (Printexc2.to_string e)
      !!O.chat_app_host !!O.chat_app_port;
      close_out chanout

With the change to the OCAML unix library and these changes to mldonkey, I no longer have any orphaned file descriptors and continue to have a high ID.

So if someone that knows the OCAML developers, maybe you could get them to change the OCAML unix library.

The other question is on Mac OS X, why is sa_family sometimes not defined in alloc_sockaddr?  I have not looked at what the value is, but it may be that the value is AF_INET6 and HAS_IPV6 is not defined in the makefile for making OCAML on Mac OS X.  It could be that if HAS_IPV6 was defined for Mac OS X when OCAML was made, that the error would not have occurred.

Shunga

Just my observation: with 2-5-28i and this patch I always got LowID´s, without it HighID´s.

I compiled the tag-2-5-29ab source that you pointed to that schlumpf provided with no changes.  After I posted the bug and the "try" suggestion to illustrate the error, I did notice the compilation warning and added the begin/end combination just as you did in the patch.  As I recall, it didn't change anything, but I'll try it once more to make sure using the tag-2-5-29ab source..

Shunga

did you try that with or without the patch i posted in the forum linked in my last comment to this bug? if without please try with it applied. (oh and no need to message me through savannah i get notified on changes of bugs i posted to)

it's in the svn repository mentioned in another thread in that forum-group. i added a link to an archive of the source, schlumpf provided.

I would like to try this but with what? I read the post at http://mldonkey.berlios.de/modules.php?name=Forums&file=viewtopic&t=3201&sid=6c52a2530f6046d72fdfbbb94c0c1d72 and I have looked at the CVS-page but I don't know where the/which source to download - where is the 29ab-version?

Programmer asleep at the switch :-).  I was wrong about some other software change.  Turns out tcp_handler does fail if changed as follows:

let tcp_handler t sock event =
  match event with
  | CAN_READ
  | CAN_WRITE ->
      try
        let s,id = Unix.accept (fd sock) in
        if !verbose_bandwidth > 1 then lprintf "[BW2 %6d] accept on %s\n" (last_time ()) t.name;
        (match t.accept_control with
            None -> () | Some cc ->
              cc.nconnections_last_second <- cc.nconnections_last_second + 1);
        incr nconnections_last_second;
        t.event_handler t (CONNECTION (s,id))
      with  e ->
        lprintf "Exception tcp_handler: %s\n" (Printexc2.to_string e);
        close t (Closed_for_error (Printexc2.to_string e));
        raise e
  | _ -> t.event_handler t (BASIC_EVENT event)

and it leaves one socket orphaned which I assume is "s".  I don't know how to get "s" down into the "with -> e" so that it can be closed and I don't know if I need to "close t" as is indicated in the code which at the moment is closing one of the server sockets that is being listened to., however, with this change "mlnet" has run for hours with only one orphaned socket.  This plus the commonChat change should get rid of the orphaned sockets.  I'll leave it up to the experts to figure out what is really going on and how to best fix it.

On Mac OS X and I assume other systems there are two bugs in mlnet which together generate hundreds of orphaned file descriptors causing mlnet to eventually hang.  I worked with mlnet 2.5-22 source and ocaml 3.07-p12 to debug the source of the problem:

I don't know the source code or OCAML well enough to suggest exactly why it is happening or the best way to fix it, but I have done enough debugging to figure out the cause of the probem. There are two issues:

1. The first is in src/daemon/common/commonChat.ml in the routine send_paquet_to_mlchat. The Unix.connect fails with "Connection refused : connect" but the error is not trapped and the socket is not closed. Trapping the error and closing the socket fixes this one.

2. The rest of the orphaned file descriptors is in src/utils/net/tcpServerSocket.ml in the routine tcp_handler. The Unix.accept fails with Exception tcp_handler: failed: Address family not supported by protocol family" but apparently has created a new socket which is never closed. If I trap the exception and issue the following "close t (Closed_for_error (Printexc2.to_string e));" I find that lsof only shows one orphaned socket after hours of running. I assume that issuing "close t" closes the original socket that is being listened to and this stop this Unix.accept from being called again. I don't know why it is getting this error unless perhaps the previous bind failed and that wasn't trapped, but maybe there is some other reason.

So if a developer who knows the code and OCAML can fix these two problems and get the patches in the current release then that should solve the orhpaned file descriptor problem which causes mlnet to hang after running for some hours